Method and apparatus for video encoding for each spatial sub-area, and method and apparatus for video decoding for each spatial sub-area

ABSTRACT

Provided are a video encoding method and a video decoding method according to spatial subdivisions based on splitting a picture into a first tile and a second tile, and splitting a current tile among the first tile and the second tile into at least one slice segment, encoding the first tile and the second tile, independently from each other, and encoding maximum coding units of a current slice segment among the at least one slice segment included in the current tile, with respect to the at least one slice segment included in the current tile.

CROSS-REFERENCE TO RELATED PATENT APPLICATION

The present application is a continuation application of U.S.application Ser. No. 14/375,649, filed on Jul. 30, 2014, in the U.S.Patent and Trademark Office, which is a national stage application under35 U.S.C. §371 of International Application No. PCT/KR2013/00754, filedon Jan. 30, 2013, and claims the benefit of U.S. Provisional ApplicationNo. 61/592,572, filed on Jan. 30, 2012, in the U.S. Patent and TrademarkOffice, the disclosures of which are incorporated herein by reference intheir entireties.

BACKGROUND

1. Field

Methods and apparatuses consistent with exemplary embodiments of thepresent application relate to encoding and decoding video according tospatial subdivisions.

2. Description of Related Art

As hardware for reproducing and storing high resolution or high qualityvideo content is being developed, there is an increasing need for avideo codec for effectively encoding or decoding the high resolution orhigh quality video content. According to conventional video codecs, avideo is encoded according to a limited encoding method based on amacroblock having a predetermined size.

Image data of a spatial domain is transformed into coefficients of afrequency domain via a frequency transformation. According to a videocodec, an image is split into blocks having a predetermined size, adiscrete cosine transformation (DCT) is performed for each respectiveblock, and frequency coefficients are encoded in block units, for rapidcalculation of a frequency transformation. Compared with image data of aspatial region, coefficients of a frequency region are easilycompressed. In particular, because an image pixel value of a spatialregion is expressed according to a prediction error via inter predictionor intra prediction of a video codec, when a frequency transformation isperformed on the prediction error, a large amount of data may betransformed to values of zero. According to a video codec, an amount ofdata may be reduced by replacing data that is consecutively andrepeatedly generated with small-sized data.

In particular, the data size of high definition or high image qualityvideo content increases, and accordingly, a need to process video afterspatially dividing the video is increasing.

SUMMARY

According to an aspect an exemplary embodiment, there is provided amethod of encoding video by spatial subdivisions, the method including:splitting a picture into a first tile and a second tile, and splitting acurrent tile among the first tile and the second tile into at least oneslice segment; encoding the first tile and the second tile, the firsttile and the second tile independently encoded from each other; andencoding maximum coding units of a current slice segment among the atleast one slice segment included in the current tile, with respect tothe at least one slice segment included in the current tile.

According to aspects of the exemplary embodiments, a relationshipbetween a tile, a slice segment, and a slice is clearly defined so thatan accessibility of maximum coding units to a reference object atboundaries between tiles, boundaries between slice segments, andboundaries between slices may be clearly regulated.

Also, because information representing whether a current slice segmentis an initial slice segment of a picture is included in a slice segmentheader and information representing whether the current slice segment isa dependent slice segment is not included in the slice segment headerwhen the slice segment is the initial slice segment, a transmission bitamount for transmitting header information may be reduced and anunnecessary parsing operation for parsing header information may beskipped.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1A is a block diagram of a video encoding apparatus encoding byspatial subdivisions, according to an exemplary embodiment;

FIG. 1B is a flowchart illustrating a video encoding method executed bythe video encoding apparatus of FIG. 1A;

FIG. 1C is a block diagram of a video encoding apparatus encoding byspatial subdivisions, according to another exemplary embodiment;

FIG. 1D is a flowchart illustrating a video encoding method executed bythe video encoding apparatus of FIG. 1C;

FIG. 2A is a block diagram of a video decoding apparatus decoding byspatial subdivisions, according to an exemplary embodiment;

FIG. 2B is a flowchart illustrating a video decoding method executed bythe video decoding apparatus of FIG. 2A;

FIG. 2C is a block diagram of a video decoding apparatus decoding byspatial subdivisions, according to another exemplary embodiment;

FIG. 2D is a flowchart illustrating a video decoding method executed bythe video encoding apparatus of FIG. 2C;

FIG. 3 is a diagram showing tiles and maximum coding units in a picture;

FIG. 4 is a diagram showing a slice segment, a slice, and maximum codingunits in a picture;

FIGS. 5A and 5B are diagrams for describing a relationship between thetiles and the slice segments in the picture;

FIGS. 6A and 6B are diagrams for describing a relationship between thetiles, the slice segments, the slice, and the maximum coding units;

FIG. 7 is a diagram showing syntax of a slice segment header accordingto an exemplary embodiment;

FIG. 8 is a block diagram of a video encoding apparatus based on codingunits according to a tree structure, according to an exemplaryembodiment;

FIG. 9 is a block diagram of a video decoding apparatus based on codingunits according to a tree structure, according to an exemplaryembodiment;

FIG. 10 is a diagram for describing a concept of coding units accordingto an exemplary embodiment;

FIG. 11 is a block diagram of an image encoder based on coding units,according to an exemplary embodiment;

FIG. 12 is a block diagram of an image decoder based on coding units,according to an exemplary embodiment;

FIG. 13 is a diagram illustrating deeper coding units according todepths, and partitions according to an exemplary embodiment;

FIG. 14 is a diagram for describing a relationship between a coding unitand transformation units, according to an exemplary embodiment;

FIG. 15 is a diagram for describing encoding information of coding unitscorresponding to a coded depth, according to an exemplary embodiment;

FIG. 16 is a diagram of deeper coding units according to depths,according to an exemplary embodiment;

FIGS. 17 through 19 are diagrams for describing a relationship betweencoding units, prediction units, and transformation units, according toan exemplary embodiment;

FIG. 20 is a diagram for describing a relationship between a codingunit, a prediction unit or a partition, and a transformation unit,according to encoding mode information;

FIG. 21 illustrates a physical structure of a disc that stores aprogram, according to an exemplary embodiment;

FIG. 22 illustrates a disc drive that records and reads a program byusing a disc;

FIG. 23 illustrates an entire structure of a content supply system thatprovides a content distribution service;

FIGS. 24 and 25 illustrate external and internal structures of a mobilephone to which a video encoding method and a video decoding method areapplied, according to an exemplary embodiment;

FIG. 26 illustrates a digital broadcasting system employing acommunication system, according to an exemplary embodiment; and

FIG. 27 illustrates a network structure of a cloud computing systemusing a video encoding apparatus and a video decoding apparatus,according to an exemplary embodiment.

DETAILED DESCRIPTION OF EXEMPLARY EMBODIMENTS

Hereinafter, video encoding and decoding methods according to exemplaryembodiments using spatial subdivisions will be described with referenceto FIGS. 1A through 7. Also, a video encoding method and a videodecoding apparatus using a quantization parameter determination methodbased on coding units having a tree structure according to exemplaryembodiments will be described with reference to FIGS. 8 through 20. Inaddition, various exemplary embodiments to which the video encoding anddecoding methods the exemplary embodiment may be applied will bedescribed with reference to FIGS. 21 through 27. Hereinafter, the term‘image’ may refer to a still image or a moving picture, that is, video.

First, referring to FIGS. 1A through 7, a video encoding method usingspatial subdivisions and a video decoding method using spatialsubdivisions according to an exemplary embodiment will be described.

FIG. 1A is a block diagram of a video encoding apparatus 101 encoding byspatial subdivisions, according to an exemplary embodiment. FIG. 1B is aflowchart illustrating a video encoding method (105) executed by thevideo encoding apparatus 101 of FIG. 1A.

The video encoding apparatus 101 according to the present exemplaryembodiment includes a sub-region divider 102 and a sub-region encoder104.

In operation S106, the sub-region divider 102 the exemplary embodimentmay divide a picture into two or more tiles and at least one slicesegment.

In operation S107, the sub-region encoder 104 the exemplary embodimentindependently encodes each tile, and may encode each slice segment.

The video encoding processes of the present exemplary embodiment may beclassified as a source encoding process, in which overlapping data dueto temporal and spatial similarity of image data is minimized, and anentropy encoding process, in which redundancy is minimized in abitstring of data generated through the source encoding process. Thesub-region encoder 104 present exemplary embodiment performs sourceencoding on each of the pictures constituting the video by block unitsto generate encoding symbols. The source encoding process includes intraprediction, inter prediction, transformation, and quantization for videodata of a spatial domain by block units. As a result of the sourceencoding process, the coding symbols may be generated in each of theblocks. For example, the coding symbols may be quantized transformationcoefficients of residual components, motion vectors, intra mode type,inter mode type, and quantization parameters.

The entropy encoding of the present exemplary embodiment may beclassified into a binarization process for transforming symbols into bitstrings, and an arithmetic coding process performing an arithmeticcoding on bit strings based on a context. Context-based adaptive binaryarithmetic coding (CABAC) is widely used as an arithmetic encodingmethod based on a context for symbol encoding. According to thecontext-based arithmetic encoding/decoding, each bit of a symbol bitstring may be a bin of a context, and a location of each bit may bemapped to a bin index. A length of the bit string, that is, a length ofthe bin, may vary according to a size of a symbol value. Contextmodeling for determining a context of a symbol is required to performthe context-based arithmetic encoding/decoding.

The context is renewed according to locations of bits of the symbol bitstring, that is, in each bin index, to perform the context modeling, andthus a complicated operation process is required. Here, the contextmodeling is a process of analyzing a probability of generating 0 or 1 ineach of the bins. A process of updating the context by reflecting aresult of analyzing the probability of symbols by bit units in newblocks to the context so far may be repeatedly performed for each block.As information including the context modeling results, a probabilitytable, in which a generation probability is matched to each bin, may beprovided. Entropy encoding probability information according to theexemplary embodiment may include the context modeling results.

Therefore, when the context modeling information, that is, the entropycoding probability information, is ensured, the entropy encoding may beperformed by allocating a code to each of the bits in binarized bitstrings of the block symbols based on the context of the entropy codingprobability information.

Also, the entropy encoding is performed by the arithmeticencoding/decoding based on the context, and the symbol code probabilityinformation may be updated in each block. Because the entropy coding isperformed by using the updated symbol code probability information, acompression rate may be improved.

The video encoding method according to various exemplary embodiments isnot limited to the video encoding method for the ‘block’, and may beused for various data units.

For efficiently performing the video encoding, the video is divided intoblocks having predetermined sizes and then the blocks of predeterminedsizes are encoded. A block may have a square shape or a rectangularshape, or may have an arbitrary geometric shape, but the block is notlimited to a data unit having a predetermined size. According to thevideo encoding method based on the coding units having a tree structure,a block may be a maximum coding unit, a coding unit, a prediction unit,a transformation unit, or the like. The video encoding/decoding methodbased on the coding units having a tree structure will be described withreference to FIGS. 8 to 20.

Blocks in a picture are encoded according to a raster scanningdirection.

The sub-region divider 102 divides a picture into one or more tiles, andeach of the tiles may include blocks arranged according to a rasterdirection among the blocks of the picture. The picture may be dividedinto tiles as one or more vertical rows, tiles as one or more horizontalrows, or tiles as one or more vertical rows and one or more horizontalrows. Each of the tiles divides the spatial region, and the sub-regionencoder 104 may encode each of the tiles independently in order toencode each of the spatial regions in operation S107.

Because each of the slice segments includes blocks arranged in theraster direction, the sub-region divider 102 may generate a slicesegment by dividing the picture in a horizontal direction. The picturemay be divided into one or more slice segments. Each of the slicesegments may be transmitted through one network adaptation layer (NAL).

The sub-region encoder 104 of the present exemplary embodiment mayperform encoding on the slice segments. The sub-region encoder 104sequentially performs the encoding on the blocks included in each of theslice segments to generate encoding symbols of the blocks. Encoding dataof the blocks may be included in one NAL unit to be transmitted in eachof the slice segments. Each of the tiles may include at least one slicesegment. If necessary, the slice segment may include at least one tile.

According to the exemplary embodiment, if the blocks of each of theslice segments are the maximum coding units including coding unitsaccording to a tree structure, a relationship between the slice segmentand the tile may satisfy one of the following conditions: (i) themaximum coding units included in one slice segment may be included inthe same tile, (ii) the maximum coding units included in one tile may beincluded in the same slice segment, and (iii) the maximum coding unitsincluded in one slice segment may be included in the same tile, and atthe same time, the maximum coding units included in one tile may beincluded in one same slice segment. Among the above conditions, if themaximum coding units included in one slice segment are included in thesame tile, it may be determined that the slice segment does not spanover boundaries of the current tile. That is, each of the slice segmentshas to be completely included in the tile. That is, a first maximumcoding unit and a last maximum coding unit among the maximum codingunits of the slice segment may be included in the same tile. Inparticular, if the first maximum coding unit of the current slicesegment is located at a center portion of the tile, the current slicesegment should not span over the boundary of the current tile.

Also, the slice segments may be classified as dependent slice segmentsand independent slice segments.

If the current slice segment is a dependent slice segment, an in-pictureprediction that refers to the encoding symbols of the previous slicesegment that is previously encoded before the current slice segment maybe performed. Also, if the current slice segment is a dependent slicesegment, a dependent entropy encoding that refers to the entropyinformation of the previous slice segment may be performed.

If the current slice segment is an independent slice segment, thein-picture prediction referring to the encoding symbols of the previousslice segment is not performed and the entropy information of theprevious slice segment is not referenced.

One slice of the present exemplary embodiment may include oneindependent slice segment and at least one dependent slice segmentsuccessive to the independent slice segment according to the rasterscanning direction. One independent slice segment may configure oneslice.

According to the exemplary embodiment, if the each slice segment and theblocks of the slice are the maximum coding units including the codingunits according to a tree structure, a relationship between the sliceand the tile may satisfy one of the following conditions: (i) themaximum coding units included in one slice are included in the sametiles, (ii) the maximum coding units included in one tile are includedin the same slices, and (iii) the maximum coding units included in oneslice are included in the same tiles, and at the same time, the maximumcoding units included in one tile may be included in the same slices.

The sub-region encoder 104 of the present exemplary embodiment mayencode each of the tiles, independently from the other tiles. Thesub-region encoder 104 may sequentially encode the maximum coding unitsincluded in the current tile, in each of the tiles.

Also, the sub-region encoder 104 of the present exemplary embodiment mayencode the maximum coding units in the current slice segment, in each ofthe slice segments. Among the maximum coding units included in thecurrent slice segment, the maximum coding units included in apredetermined tile may be encoded according to an encoding order in thecurrent tile.

The sub-region encoder 104 of the present exemplary embodiment mayencode a plurality of maximum coding units included in the current slicesegment according to the raster scanning order in the current tile, whenall the maximum coding units of the current slice segment are includedin the current tile. In this case, because the current slice segmentdoes not span over the boundary of the current tile, the maximum codingunits of the current slice segment are not located beyond the boundaryof the current tile. In this case, the sub-region encoder 104 of thepresent exemplary embodiment may sequentially encode the at least oneslice segment included in each tile, and may encode the plurality ofblocks included in each of the slice segments according to the rasterscanning order.

Also, in a case where the current slice segment includes at least onetile, the sub-region encoder 104 may encode the maximum coding unitsincluded in the current tile among the maximum coding units included inthe current slice segment, according to the raster scanning order of themaximum coding units in the current tile. The sub-region encoder 104 ofthe present exemplary embodiment may sequentially encode the slicesegments. Therefore, the sub-region encoder 104 of the present exemplaryembodiment sequentially encodes the slice segments, and sequentiallyencodes the blocks included in each of the slice segments to generateencoding symbols of the blocks. In each of the blocks in the slicesegment, intra prediction, inter prediction, transformation, in-loopfiltering, sampling adaptive offset (SAO) compensation, and quantizationmay be performed.

The sub-region encoder 104 of the present exemplary embodiment performsthe entropy encoding by using the encoding symbols generated in theblocks in each of the slice segments. The blocks included in each of theslice segments may be sequentially entropy encoded.

For performing prediction encoding on the encoding symbols generatedduring the source encoding process, for example, the intra sample, themotion vector, and the encoding mode information, in-picture predictionmay be performed. In a case where the in-picture prediction isperformed, a difference value between the current encoding symbol andthe previous encoding symbol, instead of the current encoding symbol,may be encoded. In addition, a difference between the current sample anda neighboring sample, instead of the current sample, may be encoded.

Also, in order to perform the prediction encoding on the entropy contextinformation or the code probability information generated during theentropy encoding process, a dependent entropy encoding may be performed.When the dependent entropy encoding is performed, the encoding of thecurrent entropy information may be skipped in a case where the currententropy information and the previous entropy information are equal toeach other.

However, because the sub-region encoder 104 may encode each of the tilesindependently, the in-picture prediction or the dependent entropyencoding may not be performed on the maximum coding units included indifferent tiles.

The video encoding apparatus 101 of the present exemplary embodiment mayinclude a central processor that controls the sub-region divider 102 andthe sub-region encoder 104. Otherwise, the sub-region divider 102 andthe sub-region encoder 104 may be driven respectively by their ownprocessors, and the processors may operate together to controloperations of the video encoding apparatus 101. Otherwise, thesub-region divider 102 and the sub-region encoder 104 may be controlledby an external processor external to the video encoding apparatus 101.

The video encoding apparatus 101 of the present exemplary embodiment mayinclude one or more data storage units in which input/output data of thesub-region divider 102 and the sub-region encoder 104 is stored. Thevideo encoding apparatus 101 may include a memory controller controllingthe input/output data of the data storage units.

When the bit stream of the slice segment that is encoded according tothe video encoding method (105) described with reference to FIGS. 1A and1B is transmitted, a slice segment header may be transmitted together.

Hereinafter, a method of transmitting the slice segment header accordingto the characteristic of the slice segment will be described below withreference to FIGS. 1C and 1D. The above-described relationship betweenthe sub-regions divided as the slice segment and blocks (maximum codingunits), the tile, and the slices, and the encoding performed on each ofthe sub-regions with reference to FIGS. 1A and 1B may be applied toFIGS. 1C and 1D.

FIG. 1C is a block diagram of a video encoding apparatus 10 encoding byspatial subdivisions, according to another exemplary embodiment. FIG. 1Dis a flowchart illustrating a video encoding method 11 executed by thevideo encoding apparatus 10 of FIG. 1C.

The video encoding apparatus 10 of the present exemplary embodimentincludes a slice segment encoder 12 and a slice segment transmitter 14.The slice segment encoder 12 and the slice segment transmitter 14 of thepresent exemplary embodiment may perform the source encoding process andthe entropy encoding process, respectively. In operation S111, the slicesegment encoder 12 may encode each of the slice segments after dividinga picture into at least one slice segment.

For example, if the blocks configuring the slice segment are the maximumcoding units, the slice segment encoder 12 of the present exemplaryembodiment may encode a plurality of maximum coding units included inthe current slice segment according to the raster scanning order in thecurrent tile. In operation S113, the slice segment transmitter 14 maygenerate a slice segment header including information representingwhether the current slice segment is an initial slice segment in thecurrent picture.

Default information about the current picture in which the current slicesegment is included may be recorded in a picture parameter set (PPS) andtransmitted. In particular, the PPS may include information representingwhether the current picture includes dependent slice segments.Therefore, when the information representing whether the current pictureincludes the dependent slice segments is recorded in the PPS, the slicesegment transmitter 14 may record information representing whether thecurrent slice segment is a dependent slice segment that uses sliceheader information of the previous slice segment in the current slicesegment header.

On the other hand, if the PPS of the current picture includesinformation representing that the dependent slice segment is notincluded in the current picture, the current slice segment header doesnot include the information representing whether the current slicesegment is a dependent slice segment.

In operation S115, the slice segment transmitter 14 may add informationrepresenting whether the current slice segment is a dependent slicesegment to the slice segment header, if the current slice segment is notthe initial slice segment.

That is, in a case where the PPS of the current picture includesinformation representing that the dependent slice segment is used in thecurrent picture and the current slice segment header includes theinformation representing that the current slice segment is not theinitial slice segment, information representing whether the currentslice segment is a dependent slice segment may be added to the currentslice segment header. According to the exemplary embodiment, the initialslice segment is an independent slice segment. Therefore, the slicesegment transmitter 14 may skip adding the information representingwhether the slice segment is the dependent slice segment into thecurrent slice segment header, if the current slice segment is theinitial slice segment. Therefore, the slice segment transmitter 14 maytransmit the slice segment header for the initial slice segment, byadding the information representing whether the slice segment is theinitial slice segment and default information about the current slicesegment to the slice segment header.

Therefore, in a case where the dependent slice segment may be used inthe current picture and the current slice segment is not the initialslice segment, the information representing whether the current slicesegment is a dependent slice segment may be added to the current slicesegment header.

However, if the current slice segment is not the initial slice segment,but the dependent slice segment, some of the default information aboutthe slice segment may be equal to that of the previous slice segmentheader information. Therefore, the current slice segment header includesthe information representing whether the current slice segment is theinitial slice segment or the dependent slice segment, and inserting ofthe information equal to that of the previous slice segment header intothe current slice segment header may be skipped.

According to the exemplary embodiment, if the current slice segment isnot the dependent slice segment, the current slice segment header mayfurther include various header information for the current slicesegment, while including the information representing whether thecurrent slice segment is a dependent slice segment.

For example, the slice segment transmitter 14 may record a quantizationparameter and initial probability information of the context for theentropy encoding in the slice segment header, and transmits the slicesegment header.

However, if the current slice segment is a dependent slice segment, theslice segment transmitter 14 may perform in-picture prediction thatrefers to the encoding symbol of the previous slice segment that isencoded before the current slice segment. When the current slice segmentis the dependent slice segment, the slice segment transmitter 14 mayperform the dependent entropy encoding that refers to the entropyinformation of the slice segment that is previously encoded.

Therefore, the slice segment transmitter 14 does not record thequantization parameter and the initial probability information in theslice segment header of the current slice header, when the current slicesegment is the dependent slice segment because the quantizationparameter and the initial probability information of the dependent slicesegment may be initialized as the quantization parameter and the initialprobability information recorded in the header information of theindependent slice segment that is previously encoded.

If the current slice segment is an independent slice segment, the slicesegment transmitter 14 may output a bit stream of the encoding symbol ofthe current slice segment without regard to the previous slice segment,because the in-picture prediction is not performed. If the current slicesegment is an independent slice segment, the slice segment transmitter14 may output entropy information of the current slice segment withoutregard to the entropy information of the neighboring slice segment thatis previously encoded. For example, if the current slice segment is anindependent slice segment, the quantization parameter and the initialprobability information are recorded in the current slice segmentheader.

In operation S117, the slice segment transmitter 14 may transmit theslice segment header and the symbols of the slice segment with respectto each of the slice segments.

The video encoding apparatus 10 of the present exemplary embodiment mayinclude a central processor that controls operations of the slicesegment encoder 12 and the slice segment transmitter 14. Otherwise, theslice segment encoder 12 and the slice segment transmitter 14 may bedriven by their own processors, and the processors may operate togetherto operate the video encoding apparatus 10. Otherwise, the slice segmentencoder 12 and the slice segment transmitter 14 may be controlled by anexternal processor external to the video encoding apparatus 10.

The video encoding apparatus 10 of the present exemplary embodiment mayinclude one or more data storage units in which input/output data of theslice segment encoder 12 and the slice segment transmitter 14 is stored.The video encoding apparatus 10 may include a memory controller forcontrolling input/output data of the data storage units.

Processes of decoding video by using the bit stream in which the dataencoded according to the spatial subdivisions, as described withreference to FIGS. 1A and 1B, will be described below with reference toFIGS. 2A and 2B. Concepts of the slice segment, the tile, and the slicedefined as the spatial subdivisions in FIGS. 1A and 1B may be applied toa video decoding process that will be described below.

FIG. 2A is a block diagram of a video decoding apparatus 201 thatdecodes according to spatial subdivisions according to an exemplaryembodiment. FIG. 2B is a flowchart illustrating a video decoding method(205) executed by the video decoding apparatus 201 of FIG. 2A.

The video decoding apparatus 201 of the present exemplary embodimentincludes a sub-region receiver 202 and a sub-region decoder 204.

In operation S206, the sub-region receiver 202 may receive a bit streamthat is generated as a result of processes of dividing a picture intotwo or more tiles and at least one slice segment and encoding thedivisions. The bit stream may be data generated in each of the slicesegments and data generated in each tile.

The sub-region receiver 202 of the present exemplary embodiment mayparse encoding symbols with respect to each slice segment from the bitstream. Also, the sub-region receiver 202 of the present exemplaryembodiment may parse encoding symbols with respect to each tile from thebit stream. Hereinafter, processes of performing a decoding operation ineach tile and the slice segments by the sub-region decoder 204 will bedescribed with reference to operations S207 and S208.

In operation S207, the sub-region decoder 204 may decode a tile by usingthe encoding symbols of the tile parsed from the bit stream. Inaddition, in operation S208, the sub-region decoder 204 of the presentexemplary embodiment may decode a current slice segment by using theencoding symbols of the slice segment parsed from the bit stream.

Finally, the sub-region decoder 204 may restore a picture by combiningthe tile and the slice segments reconstructed in operations S207 andS208.

When the sub-region decoder 204 of the present exemplary embodimentreceives each of the slice segments through one NAL unit, encoding dataof the blocks may be included in each of the slice segments. Accordingto the present exemplary embodiment, each of the tiles may include atleast one slice segment. If necessary, the slice segment may include atleast one tile.

According to the exemplary embodiment, if blocks in each of the slicesegments are the maximum coding units including the coding unitsaccording to a tree structure, a relationship between the slice segmentand the tile may satisfy one of the following conditions: (i) themaximum coding units included in one slice segment may be included inthe same tiles, (ii) the maximum coding units included in one tile maybe included in the same slice segments, and (iii) the maximum codingunits included in one slice segment may be included in the same tilesand the maximum coding units included in one tile may be included in thesame slice segments. Among the above conditions, if (i) the maximumcoding units included in one slice segment are included in the sametiles, the slice segment may be determined not to span over a boundaryof the current tile. That is, each of the slice segments should becompletely included in the tile. That is, a first maximum coding unitand a last maximum coding unit among the maximum coding units in theslice segment may be included in the same tile. In particular, if thefirst maximum coding unit of the current slice segment is located at acenter of the tile, the current slice segment should not span over theboundary of the current tile.

According to the exemplary embodiment, if the each slice segment and theblocks of the slice are the maximum coding units including the codingunits according to a tree structure, a relationship between the sliceand the tile may satisfy one of the following conditions: (i) themaximum coding units included in one slice are included in the sametiles, (ii) the maximum coding units included in one tile are includedin the same slices, and (iii) the maximum coding units included in oneslice are included in the same tiles, and at the same time, the maximumcoding units included in one tile may be included in the same slices.

The sub-region decoder 204 of the present exemplary embodiment maydecode each of the tiles, independently from the other tiles. In one NALunit, the maximum coding units included in the current tile may besequentially decoded.

The sub-region decoder 204 of the present exemplary embodiment mayperform entropy decoding on the maximum coding units in each of theslice segments and the tiles to parse the encoding symbols for each ofthe maximum coding units. The maximum coding units included in the slicesegment and the tile are sequentially entropy decoded to parse theencoding symbols for each of the maximum coding units.

Therefore, the sub-region decoder 204 of the present exemplaryembodiment may decode the maximum coding units in the current slicesegment. The sub-region decoder 204 may sequentially decode the maximumcoding units according to the raster scanning direction by using theencoding symbols of the maximum coding units that are parsed in each ofthe slice segments.

Also, the maximum coding units included in a predetermined tile amongthe maximum coding units included in the current slice segment may bedecoded according to a decoding order in the current tile.

The sub-region decoder 204 of the present exemplary embodiment maydecode a plurality of maximum coding units included in the current slicesegment according to the raster scanning order of the current tile, whenthe entire maximum coding units of the current slice segment areincluded in the current tile. In this case, the current slice segmentdoes not span over the boundary of the current tile. The sub-regiondecoder 204 of the present exemplary embodiment sequentially decodes theat least one slice segment included in each tile, and may decode theplurality of maximum coding units included in the slice segmentaccording to the raster scanning order.

Also, if the current slice segment includes at least one tile, thesub-region decoder 204 may decode the maximum coding units included inthe current tile, among the maximum coding units included in the currentslice segment, according to the raster scanning order of the maximumcoding units in the current tile.

An in-picture prediction may be performed by using encoding symbols,such as an intra sample, a motion vector, and encoding mode informationparsed with respect to each of the maximum coding units. Through thein-picture prediction, a reconstructed value of the current encodingsymbol may be determined by combining a reconstructed value of theprevious encoding symbol and a difference between the current encodingsymbol and the previous encoding symbol. Also, a reconstructed value ofthe current sample may be determined by combining a reconstructed valueof a neighboring sample that is previously reconstructed and adifference between the current sample and the previous sample.

The decoding operation using the encoding symbols of the maximum codingunits may be performed through an inverse-quantization, aninverse-transformation, and an intra prediction/motion compensation. Forexample, the inverse-quantization of the encoding symbols of eachmaximum coding unit is performed to reconstruct transformationcoefficients of transformation units, and the transformationcoefficients of the transformation units are inverse-transformed toreconstruct residual information of prediction units. An intraprediction may be performed by using the intra sample in the residualinformation. Also, samples of the current prediction unit may bereconstructed through a motion compensation, in which anotherreconstructed prediction unit designated by the motion vector and theresidual information are combined. In addition, the SAO compensation andin-loop filtering may be performed on the maximum coding units.

Therefore, the sub-region decoder 204 of the present exemplaryembodiment may sequentially decode the maximum coding units in each ofthe slice segments and each of the tiles according to the decoding orderin the tile.

According to the exemplary embodiment, if the tile includes at least oneslice segment, the maximum coding units in each slice segment aredecoded to reconstruct each slice segment, and then, the reconstructedresults are combined to reconstruct one tile.

Also, according to the exemplary embodiment, if the slice segmentincludes at least one tile, the maximum coding units of each tile aredecoded to reconstruct the tile, and the reconstructed results of thetiles are combined to reconstruct the slice segment.

The sub-region decoder 204 of the present exemplary embodiment mayreconstruct a picture consisting of the reconstructed tiles or thereconstructed slice segments.

Processes of decoding video by using the bit stream in which dataencoded according to the spatial subdivisions as described withreference to FIGS. 1C and 1D will be described below with reference toFIGS. 2C and 2D. Concepts of the slice segments, tiles, and the slicedefined as the spatial subdivisions in FIGS. 1C and 1D may be applied tothe video decoding method that will be described below.

When receiving the bit stream of the slice segments that are decodedaccording to the video decoding method (205) described with reference toFIGS. 2A and 2B, slice segment headers may be received together.Hereinafter, processes of decoding video by using a slice segment headerwill be described below with reference to FIGS. 2C and 2D.

FIG. 2C is a block diagram of a video decoding apparatus 20 decoding byspatial subdivisions, according to another exemplary embodiment. FIG. 2Dis a flowchart of a video decoding method (21) executed by the videodecoding apparatus 20 of FIG. 2C.

The video decoding apparatus 20 of the present exemplary embodimentincludes a slice segment parser 22 and a slice segment decoder 24. FIGS.2C and 2D illustrate the slice segments; however, a relationship betweenthe slice segment and the tile, and the slice segment are describedabove with reference to FIGS. 2A and 2B.

In operation S211, the video decoding apparatus 20 of the presentexemplary embodiment may receive the bit stream that is generated byencoding the picture in the slice segment unit. The bit stream of eachslice segment may include a slice segment header and encoding symbols.The slice segment parser 22 according to the present exemplaryembodiment may receive the bit stream of each of the slice segments,wherein the bit stream includes the slice segment header and thesymbols. The slice segment parser 22 of the present exemplary embodimentmay parse the symbols of the current slice segment in the bit stream.The slice segment parser 22 according to the present exemplaryembodiment parses the slice segment header of the current slice segmentin the bit stream, and may parse various pieces of header informationabout the current slice segment from the slice segment header.

Hereinafter, a method of parsing the slice segment header by the slicesegment parser 22 according to the characteristics of the slice segmentwill be described below with reference to operations S213 through S217.

In operation S213, the slice segment parser 22 of the present exemplaryembodiment may parse information representing whether the current slicesegment is an initial slice segment in the current picture from theslice segment header of the current slice segment.

If the slice segment parser 22 determines that the current slice segmentis not the initial slice segment from the parsed information, theprocess goes to operation S215.

In operation S215, if the current slice segment is not the initial slicesegment, the slice segment parser 22 may further parse informationrepresenting whether the current slice segment is a dependent slicesegment using the slice header information of the previous slicesegment, from the current slice segment header.

However, the information representing whether the current pictureincludes the dependent slice segment may be parsed from PPS about thecurrent picture including the current slice segment. Therefore, in acase where the information representing that the current pictureincludes the dependent slice segment is parsed from the PPS of thecurrent picture, the slice segment parser 22 may parse informationrepresenting whether the current slice segment is a dependent slicesegment, from the current slice segment header.

On the other hand, if the information representing that the currentpicture does not use the dependent slice segment is parsed from the PPSof the current picture, the information representing whether the currentslice segment is a dependent slice segment is not parsed from thecurrent slice segment header.

Therefore, if the information representing that the current pictureincludes the dependent slice segment is parsed from the PPS of thecurrent picture and the information representing that the current slicesegment is not the initial slice segment is parsed in operation S213,the slice segment parser 22 may further parse information representingwhether the current slice segment is a dependent slice segment, from thecurrent slice segment. That is, if the current picture includes thedependent slice segment and the current dependent slice segment is notthe initial slice segment, the information representing whether thecurrent slice segment is a dependent slice segment may be further parsedfrom the current slice segment header.

In operation S213, if the slice segment parser 22 determines that thecurrent slice segment is the initial slice segment from the parsedinformation, the information representing whether the current slicesegment is a dependent slice segment is not parsed from the currentslice segment header. Because the initial slice segment cannot be thedependent slice segment, the initial slice segment may be determined asan independent slice segment without using the parsing information.Therefore, if the current slice segment is the initial slice segment,the slice segment parser 22 of the present exemplary embodiment mayfurther parse information representing whether the slice segment is theinitial slice segment and default information about the current slicesegment from the initial slice segment header of the picture.

If the slice segment parser 22 reads that the current slice segment isthe dependent slice segment from the information parsed from the currentslice segment header, the slice segment parser 22 may determine some ofthe header information parsed from the previous slice segment header asdefault information of the current slice segment.

If the slice segment parser 22 determines that the current slice segmentis not the dependent slice segment from the information parsed from thecurrent slice segment header, the slice segment parser 22 may parsevarious pieces of header information of the current slice segment fromthe current slice segment header.

In operation S217, the slice segment decoder 24 of the present exemplaryembodiment may decode the current slice segment by using the informationparsed from the current slice segment header and the symbols of thecurrent slice segment.

Also, the slice segment decoder 24 of the present exemplary embodimentmay reconstruct at least one slice segment included in each of thetiles, including the current slice segment reconstructed through thedecoding operation in operation S217, and may restore the picture bycombining the reconstructed tiles.

The slice segment parser 22 of the present exemplary embodiment mayparse symbols of the plurality of blocks included in the current slicesegment according to a raster scanning order, in each of the slicesegments included in each of the tiles. Also, the slice segment decoder24 of the present exemplary embodiment may decode the blocks accordingto the raster scanning order by using the symbols of the blocks, whichare parsed according to the raster scanning order of the blocks.

The slice segment parser 22 of the present exemplary embodiment mayperform entropy decoding on the bit stream of each of the slice segmentsto parse the encoding symbols for each of the maximum coding units. Themaximum coding units included in the slice segment are sequentiallyentropy decoded to parse the encoding symbols of each of the maximumcoding units.

Therefore, the slice segment decoder 24 of the present exemplaryembodiment may perform the decoding of each of the maximum coding unitssequentially according to the raster scanning order, by using the parsedencoding symbols of the maximum coding units in each of the slicesegments.

Therefore, the slice segment decoder 24 of the present exemplaryembodiment may sequentially decode the maximum coding units in each ofthe slice segments to reconstruct each slice segment, and mayreconstruct the picture consisting of the reconstructed slice segments.

As described above with reference to FIGS. 1A through 2B, the picturemay be divided into tiles or slice segments. A tile is a data unit forencoding or decoding the picture independently in each spatialsubdivision unit, and the slice segment is a unit divided fortransferring data. Therefore, during encoding or decoding tiles,encoding information of other tiles may not be referred to at a boundarybetween adjacent tiles. However, in the encoding or decoding processesof the slice segments, encoding information of other slice segments maybe selectively referred to at a boundary between the adjacent slicesegments.

Therefore, because characteristics of the slice segment and the tile aredifferent from each other in performing the prediction encoding, theremay be a problem when the slice segment and the tile spatially overlapeach other. For example, if one slice segment includes a boundarybetween the tiles, blocks of the same slice segment may be located indifferent tiles based on the boundary between the tiles. In this case,it is unclear whether the blocks crossing over the boundary between thetiles may be encoded or decoded by referring to each other.

Therefore, according to the video encoding apparatus 101 and the videodecoding apparatus 201 of the present exemplary embodiment,relationships between the tile, the slice segment, and the slice areclearly defined so that accessibility to the references of the maximumcoding units at boundaries between the tiles, between the slicesegments, and between the slices may be clearly regulated.

Also, because the initial slice segment is always an independent slicesegment, there is no need to determine whether the initial slice segmentis the dependent slice segment. Therefore, according to the videoencoding apparatus 10 and the video decoding apparatus 20 described withreference to FIGS. 1C, 1D, 2C, and 2D, information representing whetherthe current slice segment is the initial slice segment of the picture isincluded in the slice segment header, and in a case of the initial slicesegment, the information representing whether the current slice segmentis a dependent slice segment may not be included in the slice segmentheader. Accordingly, a transmission bit amount for transferringunnecessary header information may be reduced, and unnecessary parsinginformation for reading the header information may be skipped.

Hereinafter, relationships between the slice segment, the tile, and theslice that are the sub-regions used in the video encoding apparatus 101and the video decoding apparatus 201 according to exemplary embodimentswill be described below with reference to FIGS. 3 through 6B.

FIG. 3 shows tiles and maximum coding units in a picture.

When performing encoding and decoding on each of the regions generatedby dividing a picture 301 in at least one of a vertical direction and ahorizontal direction, each of the regions may be referred to as a tile.In order to process a large amount of data of a high-definition (HD) orultra high-definition (UHD) video in real-time, the picture 301 may bedivided into at least one column and at least one row to generate tiles,and the encoding and decoding may be performed with on each of thetiles.

In the picture 301, because each of the tiles is a spatial region thatis independently encoded or decoded, the tile at a desired region may beselectively encoded or decoded.

In FIG. 3, the picture 301 may be divided into the tiles by columnboundaries 321 and 323 and row boundaries 311 and 313. A regionsurrounded by one of the column boundaries 321 and 323 and one of therow boundaries 311 and 313 includes the tiles.

If the picture 301 is divided into tiles to be encoded, informationabout locations of the column boundaries 321 and 323 and the rowboundaries 311 and 313 may be recorded in a sequence parameter set (SPS)or a PPS and transmitted. When decoding the picture 301, the informationabout the locations of the column boundaries 321 and 323 and the rowboundaries 311 and 313 is parsed from the SPS or the PPS to decode eachof the tiles, and then, each of the sub-regions of the picture 301 isreconstructed, and the sub-regions may be reconstructed as the picture301 by using the information about the column boundaries 321 and 323 andthe row boundaries 311 and 313.

The picture 301 is divided into the maximum coding units (largest codingunit, LCU), and the encoding is performed on each of the blocks.Therefore, each of the tiles that are generated by dividing the picture301 by the column boundaries 321 and 323 and the row boundaries 311 and313 may include the maximum coding units. The column boundaries 321 and323 and the row boundaries 311 and 313 dividing the picture 301 extendalong boundaries between neighboring maximum coding units, and thus, donot divide the maximum coding units. Therefore, each of the tiles mayinclude an integer number of maximum coding units.

Therefore, the encoding or decoding may be performed on each of themaximum coding units in each of the tiles, while performing processeswith respect to each of the tiles in the picture 301. In FIG. 3, numericcharacters representing the maximum coding units denote a scanning orderof the maximum coding units in the tile, that is, a processing order forbeing encoded or decoded.

The tile may be compared with the slice segment and the slice, in viewthat the encoding and decoding of the tiles are independently performedfrom each other. Hereinafter, the slice segment and the slice will bedescribed below with reference to FIG. 4.

FIG. 4 shows the slice segment, the slice, and the maximum coding unitsin a picture 401.

The picture 401 is divided into a plurality of maximum coding units. InFIG. 4, the picture 401 is divided into 117 maximum coding units, thatis, 13 maximum coding unit in a horizontal direction and 9 maximumcoding units in a vertical direction. Each of the maximum coding unitsmay be divided into the coding units having a tree structure to beencoded or decoded.

The picture 401 is divided into two slices by a boundary line 411. Inaddition, the picture 401 is divided into slice segments 431, 433, 435,and 441 by boundary lines 421, 423, and 411.

The slice segments 431, 433, 435, and 441 may be classified as dependentslice segments and independent slice segments. In a dependent slicesegment, information used or generated during source encoding andentropy encoding processes of a predetermined slice segment may bereferred to in order to perform the source encoding and the entropyencoding of the other slice segments. Likewise, parsing information inthe entropy decoding process of a predetermined slice segment among thedependent slice segments and the information used or reconstructed inthe source decoding may be referred to in order to perform the entropydecoding and the source decoding on the other slice segments.

In an independent slice segment, the information used or generated inthe source encoding and the entropy encoding of each slice segment isnot referenced, and the independent slice segment is independentlyencoded. Likewise, parsing information and reconstructed information ofother slice segments are not used at all in the entropy decoding and thesource decoding of the independent slice segment.

Information representing whether the slice segment is a dependent slicesegment or an independent slice segment may be recorded in a slicesegment header and transmitted. When decoding the picture 401,information about the slice segment type is parsed from the slicesegment header, and it may be determined whether the current slicesegment will be reconstructed by referring to other slice segments ordecoded independently from the other slice segments according to thetype of the slice segment.

In particular, values of syntax elements of the slice segment header inthe independent slice segment, that is, header information, may not bededuced from the header information of the previous slice segment.However, the header information of the slice segment header in thedependent slice segment may be deduced from the header information ofthe previous slice segment.

Each of the slices may include an integer number of maximum codingunits. Also, one slice may include at least one slice segment. If oneslice includes only one slice segment, the slice segment may be anindependent slice segment. Also, one slice may include one independentslice segment and at least one dependent slice segment next to theindependent slice segment. The at least one slice segment included inone slice may be transmitted or received via the same access unit.

An upper slice of the picture 401 includes one independent slice segment431 and two dependent slice segments 433 and 435. A lower slice of thepicture 401 includes only one independent slice segment 441.

FIGS. 5A and 5B show a relationship between the tile and the slicesegment in a picture 50.

Referring to FIG. 3, the picture 301 is divided by the column boundaries321 and 323 and the row boundaries 311 and 313 to form the tiles.However, as shown in FIG. 5A, the picture 50 may be divided only bycolumn boundaries 51 and 53. That is, the picture 50 is divided by twocolumn boundaries 51 and 53 to generate three tiles, that is, a tile #1,a tile #2, and a tile #3. Also, the tile #1 may be divided by two rowboundaries 52 and 54 to form three slice segments 511, 513, and 515.

That is, the slice segments are generated by dividing the tile in ahorizontal direction, and the tiles are generated by dividing thepicture 50 in a vertical direction.

Each of the slice segments 511, 513, and 515 includes an integer numberof maximum coding units. In addition, each of the slice segments 511,513, and 515 may be obtained by dividing the current tile to include themaximum coding units arranged according to the scanning order of themaximum coding units in the current tile. The maximum coding units ineach of the slice segments 511, 513, and 515 may be included in one NALunit. Therefore, each of the slice segments 511, 513, and 515 may beindependently transmitted or received and encoded or decoded.

An inclusion relation between the tiles and the slice segments will bedescribed below with reference to FIG. 5B. A picture 525 is divided intotwo tiles #1 and #2, and three slice segments 0, 1, and 2. Because anin-picture prediction and a dependent entropy encoding referring to eachother may not be performed between different tiles, the tile #2 may notrefer to encoding symbols and entropy information of the tile #1 beyondthe boundary of the tiles.

However, the slice segments 0, 1, and 2 of the picture 525 need to referto encoding symbols and entropy information of other maximum codingunits while encoding the maximum coding units thereof according to thescanning order. Because the slice segment 1 spans over the tile #1 andthe tile #2, some of the maximum coding units of the slice segment 1 maynot refer to encoding symbols or entropy information of the othermaximum coding units included in different tiles. Therefore,configurations of the tiles #1 and #2 and the slice segment 1 in thepicture 525 are not appropriate.

A picture 535 includes two tiles #3 and #4 and four slice segments 3, 4,5, and 6. Also, the tile #3 includes two slice segments 3 and 4, and thetile #4 includes two slice segments 5 and 6.

The slice segments 3 and 4 are completely embedded in the tile #3, andthe slice segments 5 and 6 are completely embedded in the tile #4.Therefore, the slice segments 3, 4, 5, and 6 do not span over theboundary between the tiles #3 and #4.

According to the structure of the picture 535 including the tiles #3 and#4 and the slice segments 3, 4, 5, and 6, the dependency of thein-picture prediction and the entropy prediction in each of the slicesegment may be ensured.

For example, because the maximum coding units of the slice segment 3 andthe maximum coding units of the slice segment 4 are included in the sametile #3, the maximum coding units may be encoded or decoded sequentiallyaccording to the scanning orders, respectively. The maximum coding unitsof the slice segments 3 and 4 may refer to encoding symbols or entropyinformation of other maximum coding units included in the same slicesegments.

Also, because the slice segments 3 and 4 in the tile #3 are encoded ordecoded in the stated order, if the slice segment 4 is a dependent slicesegment, the slice segment 4 may be encoded or decoded by using encodingsymbols or entropy information of the slice segment 3.

The tiles #3 and #4 may be independently encoded or decoded from eachother, without referring to each other.

Also, the slice segments 5 and 6 may be encoded or decoded in the statedorder in the tile #4. Because the maximum coding units of the slicesegment 5 and the maximum coding units of the slice segment 6 areincluded in the same tile #4, each of the slice segment 5 and 6 may beencoded or decoded according to the scanning order. The maximum codingunits of the slice segments 5 and 6 may refer to encoding symbols orentropy information of other maximum coding units included in the sameslice segments.

Because the slice segment 5 is an independent slice segment that may notrefer to the tile #3, the slice segment 5 may be entropy encoded orentropy decoded by using initialized entropy information. If the slicesegment 6 is a dependent slice segment, the slice segment 6 may beentropy encoded or entropy decoded by using entropy information of theslice segment 5. In a case of the in-picture prediction, the slicesegment 5 is independently processed, and the slice segment 6 may referto information of the slice segment 5.

FIGS. 6A and 6B show relationships between the tile, the slice segment,and the maximum coding units.

The slice segment included in the current tile may be located not tospan over the boundary of the current tile. That is, the slice segmentmay be included in the tile.

Also, the slice including one independent slice segment or oneindependent slice segment and at least one dependent slice segment maybe located so that the slice segment included therein does not span overthe boundary of the current tile including the corresponding slicesegment. That is, the slice may be included in the tile.

However, if the slice or the slice segment completely includes thecurrent tile, it may be allowed that the slice or the slice segment isgreater than the current tile.

If the coding units configuring each of the slice segments, each of thetiles, and each of the slices are maximum coding units including codingunits according to a tree structure, relations between the slicesegment, the slice, and the tile may be defined by using the maximumcoding units, as follows:

The maximum coding units that are encoded (decoded) according to thescanning order in each of the tiles and in each of the slice segmentsshould satisfy one of the following conditions:

(i) the maximum coding units included in one slice segment may beincluded in the same tile;

(ii) the maximum coding units included in one tile may be included inthe same slice segment; and

(iii) the maximum coding units included in one slice segment may beincluded in the same tile, and the maximum coding units included in onetile may be included in the same slice segment.

In addition, the maximum coding units that are encoded (decoded)according to the raster scanning order in each of the slices and each ofthe tiles should satisfy one of the following conditions:

(a) the maximum coding units included in one slice may be included inthe same tile;

(b) the maximum coding units included in one tile may be included in thesame slice; and

(c) the maximum coding units included in one slice may be included inthe same tile, and the maximum coding units included in one tile may beincluded in the same slice.

Referring to FIG. 6A, a picture 60 is divided into five slice segments611, 613, 615, 617, and 619 by slice segment boundaries 603, 605, 607,and 609. Also, a slice is configured by one independent slice segment611 and four dependent slice segments 613, 615, 617, and 619, and thus,the picture 60 includes one slice.

Also, the picture 60 is divided into two tiles by a tile boundary 601.Accordingly, a left tile includes three slice segments 611, 613, and615, and a right tile includes two slice segments 617 and 619.

It will be considered whether the maximum coding units in the slicesegments 611, 613, 615, 617, and 619 and the tiles satisfy one of theconditions i, ii, and iii as follows. Because the maximum coding unitsof the slice segments 611, 613, and 615 are included in the left tile,the maximum coding units satisfy condition (i). Also, the maximum codingunits of the slice segments 617 and 619 are included in the right tile,the maximum coding units satisfy condition (i).

Then, it will be considered whether the maximum coding units of theslice and the tile satisfy one of the conditions a, b, and c, asfollows: Because the maximum coding units of the left tile are includedin one slice, condition (b) is satisfied. Also, because the maximumcoding units in the right tile are included in one slice, condition (b)is satisfied.

Referring to FIG. 6B, a picture 65 is divided into two tiles, that is, aleft tile and a right tile by a tile boundary 651. Also, the picture 65is divided into three slices by slice boundaries 66 and 68, and thus,the left tile is divided into an upper left slice and a lower left slicebased on the slice boundary 66, and the right tile may configure oneright slice.

The upper left slice may be divided into one independent slice segment661 and one dependent slice segment 665 based on the slice segmentboundary 663. The lower left slice may be divided into one independentslice segment 681 and one dependent slice segment 685 based on a segmentboundary 683. The right slice may be divided into one independent slicesegment 691 and one dependent slice segment 695 based on a slice segmentboundary 693.

It will be considered whether the maximum coding units in the slicesegments 661, 665, 681, 685, 691, and 695 and the tiles satisfy one ofthe conditions i, ii, and iii, as follows: Because the maximum codingunits of the slice segments 661 and 665 are included in the left tile,the maximum coding units satisfy condition (i). Also, the maximum codingunits of the slice segments 681 and 685 are included in the left tile,the maximum coding units satisfy condition (i). In addition, the maximumcoding units of the slice segments 691 and 695 are included in the righttile, condition (i) is satisfied.

Then, it will be considered whether the maximum coding units of theslice and the tile satisfy one of the conditions a, b, and c, asfollows: Because the maximum coding units of the upper left slice areincluded in the left tile, condition (a) is satisfied. Also, because themaximum coding units in the lower left slice are included in the lefttile, condition (a) is satisfied. In addition, because the maximumcoding units of the right slice are included in the right tile and themaximum coding units of the right tile are included in the right slice,condition (c) is satisfied.

Hereinafter, a slice segment header used by the video encoding apparatus10 and the video decoding apparatus 20 described with reference to FIGS.1C, 1D, 2C, and 2D will be described below with reference to FIG. 7.

FIG. 7 shows syntax of a slice segment header 70 according to anexemplary embodiment.

The video encoding apparatus 10 of the exemplary embodiment may generatethe slice segment header 70 including various pieces of headerinformation about the current slice segment. For example, the slicesegment header 70 may include default information required to decode thepictures included in the current slice segment, for example, current PPSidentification information, information about the number of picturesincluded in the current slice segment, information about the number ofreference pictures of the pictures, and information whether to use adifference motion vector.

The video encoding apparatus 10 according to the present exemplaryembodiment may record information 71 representing whether the currentslice segment is an initial slice segment in the current picture, in theslice segment header 70.

The video decoding apparatus 10 of the present exemplary embodiment mayadd information 75 representing whether the current slice segment is adependent slice segment to the slice segment header 70, according towhether the information 71 represents that the current slice segment isnot the initial slice segment 73. For example, if the information 71represents that the current slice segment is not the initial slicesegment, the information 75 representing whether the current slicesegment is the dependent slice segment may be added to the slice segmentheader 70.

Because the initial slice segment is an independent slice segmentaccording to the exemplary embodiment, if the current slice segment isthe initial slice segment, the information representing whether thecurrent slice segment is a dependent slice segment does not need to betransferred. Therefore, the video encoding apparatus 10 does not need toadd the information 75 next to the information representing whether theslice segment is the initial slice segment, but adds default informationabout the current slice segment to the slice segment header 70 andtransmits the slice segment header 70.

However, if the current slice segment is not the initial slice segment,but the current slice segment is a dependent slice segment, the videoencoding apparatus 10 may transmit the slice segment header 70 afterskipping some of the default information about the slice segment.

If the current slice segment is neither the initial slice segment northe dependent slice segment, that is, if the current slice segment is anindependent slice segment, the video encoding apparatus 10 may recordthe default information about the current slice segment in the slicesegment header 70 before transmitting the slice segment header 70.

Processes of parsing the slice segment header 70 by the video decodingapparatus 20 according to the exemplary embodiment are similar to thoseof generating the slice segment header by the video encoding apparatus10.

The video decoding apparatus 20 of the present exemplary embodiment mayparse the information 71 representing whether the current slice segmentis the initial slice segment in the current picture from the slicesegment header 70.

The video decoding apparatus 20 determines whether the information 71represents that the current slice segment is not the initial slicesegment 73. Based on the information 71, if it is determined that thecurrent slice segment is the initial slice segment 73, a process ofparsing the information 75 representing whether the current slicesegment is the dependent slice segment may be skipped, and then, otherheader information may be further parsed.

However, if it is determined that the current slice segment is not theinitial slice segment 73 based on the information 71, the information 75representing whether the current slice segment is the dependent slicesegment may be further parsed from the slice segment header 70.

If it is determined that the current slice segment is not the initialslice segment, but the current slice segment is the dependent slicesegment based on the information 71 and the information 75, the videodecoding apparatus 20 may parse some of the default information aboutthe current slice segment only from the current slice segment header 70.The video decoding apparatus 20 may determine the information that isnot included in the current slice segment header 70 by using theinformation acquired from the previous slice segment header.

If it is determined that the current slice segment is not the initialslice segment, but the current slice segment is the independent slicesegment based on the information 71 and the information 75, the videodecoding apparatus 20 may parse all the default information about thecurrent slice segment from the slice segment header 70.

However, the slice segment header 70 shown in FIG. 7 is obtained in acase where the PPS of the current picture includes informationrepresenting that the current picture may include dependent slicesegments. As described above with reference to FIGS. 1B and 2B, onlywhen the PPS of the current picture including the current slice segmentincludes information representing that the current picture may includethe dependent slice segments, the slice segment header 70 may includethe information 75 representing whether the current slice segment is thedependent slice segment.

Therefore, only when the information representing that the currentpicture may include the dependent slice segments is parsed from the PPSof the current picture and the information 71 parsed from the slicesegment header 70 represents that the current slice segment is not theinitial slice segment, the information 75 representing whether thecurrent slice segment is the dependent slice segment may be furtherparsed from the slice segment header 70. Therefore, the video is encodedaccording to the spatial subdivisions including the slice segments, thetiles, and the slices satisfying the above described conditions, andaccordingly, the maximum coding units configuring the tile may beincluded in the slice segment. Because the maximum coding units includedin the current slice segment are decoded according to the scanning orderof the maximum coding units in the tile, the current slice segment maybe decoded.

Also, in a case of the slice segments included in one tile, afterdecoding the independent slice segment, the dependent slice segments maybe decoded by using the decoding result of the independent slicesegment. When the independent slice segment is decoded, the entropydecoding or the in-picture prediction is not performed by referring tothe other slice segments that are located beyond the slice segmentboundary. Thus, a process of identifying the reference probability ofthe entropy information, the encoding symbols, and the samples acquiredaround the slice segment boundary for performing the entropy decoding orthe in-picture prediction may be skipped. Therefore, there is no need toinclude information for identifying the prediction probability betweenthe adjacent blocks at the slice segment boundary in the slice segmentheader 70.

Because the slice segments included in the current tile are sequentiallydecoded, the current tile may be decoded. Each of the tiles may beindependently decoded from each other. The picture may be reconstructedas a result of combining the reconstructed tiles.

When decoding the tile, the entropy decoding or the in-pictureprediction is not performed by referring to other tiles located beyondthe tile boundary, and thus, a process of identifying the referenceprobability of the entropy information, the encoding symbols, and thesamples acquired around the tile boundary for performing the entropydecoding or the in-picture prediction may be skipped. Therefore, theslice segment header 70 does not need to include information foridentifying the prediction probability between the adjacent blocks atthe tile boundary.

Also, information representing whether in-loop filtering is performed onthe tile boundary and information representing whether in-loop filteringis performed on the slice segment boundary may be selectively includedin the slice segment header 70.

Also, a location or address of the independent slice segment may beidentified through the slice segment header of the independent slicesegment. An entry point of a tile may be determined at a portion wherethe location (address) of the independent slice segment and the location(address) of the tile match each other, and thus, there is no need totransmit or parse information about the entry point of the tile.

In the video encoding apparatus 10 and the video decoding apparatus 20,blocks obtained by dividing the video data are the maximum coding units,and each of the maximum coding units is divided into coding units of atree structure, as described above. Hereinafter, a video encodingapparatus and method and a video decoding apparatus and method based onmaximum coding units and coding units of a tree structure will bedescribed below with reference to FIGS. 8 through 20.

FIG. 8 is a block diagram of a video encoding apparatus 100 based oncoding units of a tree structure, according to an exemplary embodiment.

The video encoding apparatus 100 using video prediction based on codingunits according to a tree structure includes a largest coding unit (LCU)splitter 110, a coding unit determiner 120, and an output unit 130.Hereinafter, for convenience of description, the video encodingapparatus 100 using video prediction based on coding units according toa tree structure is referred to as ‘the video encoding apparatus 100’.

The LCU splitter 110 may split a current picture based on a maximumcoding unit for the current picture of an image. If the current pictureis larger than the maximum coding unit, image data of the currentpicture may be split into at least one maximum coding unit. The maximumcoding unit according to an exemplary embodiment may be a data unithaving a size of 32×32, 64×64, 128×128, 256×256, etc., wherein a shapeof the data unit is a square having a width and length of 2^(n).

A coding unit according to an exemplary embodiment may be characterizedby a maximum size and a depth. The depth denotes the number of times thecoding unit is spatially split from the maximum coding unit, and as thedepth deepens, deeper encoding units according to depths may be splitfrom the maximum coding unit to a minimum coding unit. A depth of themaximum coding unit is an uppermost depth and a depth of the minimumcoding unit is a lowermost depth. Because a size of a coding unitcorresponding to each depth decreases as the depth of the maximum codingunit deepens, a coding unit corresponding to an upper depth may includea plurality of coding units corresponding to lower depths.

As described above, the image data of the current picture is split intothe maximum coding units according to a maximum size of the coding unit,and each of the maximum coding units may include deeper coding unitsthat are split according to depths. Because the maximum coding unitaccording to an exemplary embodiment is split according to depths, theimage data in a spatial domain included in the maximum coding unit maybe hierarchically classified according to depths.

A maximum depth and a maximum size of a coding unit, which limit thetotal number of times a height and a width of the maximum coding unitare hierarchically split, may be predetermined.

The coding unit determiner 120 encodes at least one split regionobtained by splitting a region of the maximum coding unit according todepths, and determines a depth to output finally encoded image dataaccording to the at least one split region. In other words, the codingunit determiner 120 determines a coded depth by encoding the image datain the deeper coding units according to depths, according to the maximumcoding unit of the current picture, and selecting a depth having thesmallest encoding error. The determined coded depth and the encodedimage data according to the determined coded depth are output to theoutput unit 130.

The image data in the maximum coding unit is encoded based on the deepercoding units corresponding to at least one depth equal to or below themaximum depth, and results of encoding the image data are compared basedon each of the deeper coding units. A depth having the smallest encodingerror may be selected after comparing encoding errors of the deepercoding units. At least one coded depth may be selected for each maximumcoding unit.

The size of the maximum coding unit is split as a coding unit ishierarchically split according to depths, and the number of coding unitsincreases. Also, even if coding units correspond to the same depth inone maximum coding unit, it is determined whether to split each of thecoding units corresponding to the same depth to a lower depth bymeasuring an encoding error of the image data of each coding unit,separately. Accordingly, even when image data is included in one maximumcoding unit, the image data is split into regions according to thedepths and the encoding errors may differ according to regions in theone maximum coding unit, and thus the coded depths may differ accordingto regions in the image data. Thus, one or more coded depths may bedetermined in one maximum coding unit, and the image data of the maximumcoding unit may be divided according to coding units of at least onecoded depth.

Accordingly, the coding unit determiner 120 may determine coding unitshaving a tree structure included in the maximum coding unit. The ‘codingunits having a tree structure’ according to an exemplary embodimentinclude coding units corresponding to a depth determined to be the codeddepth, from among all deeper coding units included in the maximum codingunit. A coding unit of a coded depth may be hierarchically determinedaccording to depths in the same region of the maximum coding unit, andmay be independently determined in different regions. Similarly, a codeddepth in a current region may be independently determined from a codeddepth in another region.

A maximum depth according to an exemplary embodiment is an index relatedto the number of times splitting is performed from a maximum coding unitto a minimum coding unit. A first maximum depth according to anexemplary embodiment may denote the total number of times splitting isperformed from the maximum coding unit to the minimum coding unit. Asecond maximum depth according to an exemplary embodiment may denote thetotal number of depth levels from the maximum coding unit to the minimumcoding unit. For example, when a depth of the maximum coding unit is 0,a depth of a coding unit, in which the maximum coding unit is splitonce, may be set to 1, and a depth of a coding unit, in which themaximum coding unit is split twice, may be set to 2. Here, if theminimum coding unit is a coding unit in which the maximum coding unit issplit four times, 5 depth levels of depths 0, 1, 2, 3, and 4 exist, andthus the first maximum depth may be set to 4, and the second maximumdepth may be set to 5.

Prediction encoding and transformation may be performed according to themaximum coding unit. The prediction encoding and the transformation arealso performed based on the deeper coding units according to a depthequal to or depths lower than the maximum depth, according to themaximum coding unit.

Because the number of deeper coding units increases whenever the maximumcoding unit is split according to depths, encoding including theprediction encoding and the transformation is performed on all of thedeeper coding units generated as the depth deepens. For convenience ofdescription, the prediction encoding and the transformation will now bedescribed based on a coding unit of a current depth, in a maximum codingunit.

The video encoding apparatus 100 may variously select a size or shape ofa data unit for encoding the image data. In order to encode the imagedata, operations, such as prediction encoding, transformation, andentropy encoding, are performed, and at this time, the same data unitmay be used for all operations or different data units may be used foreach operation.

For example, the video encoding apparatus 100 may select, not only acoding unit for encoding the image data, but also a data unit differentfrom the coding unit to perform the prediction encoding on the imagedata in the coding unit.

In order to perform prediction encoding on the maximum coding unit, theprediction encoding may be performed based on a coding unitcorresponding to a coded depth, i.e., based on a coding unit that is nolonger split into coding units corresponding to a lower depth.Hereinafter, the coding unit that is no longer split and becomes a basisunit for prediction encoding will now be referred to as a ‘predictionunit’. A partition obtained by splitting the prediction unit may includea prediction unit or a data unit obtained by splitting at least one of aheight and a width of the prediction unit. The partition is a data unitobtained by dividing the prediction unit of the coding unit, and theprediction unit may be a partition having the same size as the codingunit.

For example, when a 2N×2N coding unit (where N is a positive integer) isno longer split and becomes a 2N×2N prediction unit, a size of apartition may be 2N×2N, 2N×N, N×2N, or N×N. Examples of a partition typeinclude symmetrical partitions that are obtained by symmetricallysplitting a height or width of the prediction unit, partitions obtainedby asymmetrically splitting the height or width of the prediction unit,such as 1:n or n:1, partitions that are obtained by geometricallysplitting the prediction unit, and partitions having arbitrary shapes.

A prediction mode of the prediction unit may be at least one of an intramode, an inter mode, and a skip mode. For example, the intra mode or theinter mode may be performed on the partition having a size of 2N×2N,2N×N, N×2N, or N×N. Also, the skip mode may be performed only on the2N×2N partition. The encoding is independently performed on oneprediction unit in a coding unit, thereby selecting a prediction modehaving the smallest encoding error.

The video encoding apparatus 100 may also perform the transformation onthe image data in a coding unit based not only on the coding unit forencoding the image data, but also based on a transformation unit that isdifferent from the coding unit. In order to perform the transformationin the coding unit, the transformation may be performed based on a dataunit having a size smaller than or equal to the coding unit. Forexample, the transformation unit for the transformation may include atransformation unit for an intra mode and a data unit for an inter mode.

Similar to the coding unit according to a tree structure according tothe present exemplary embodiment, the transformation unit in the codingunit may be recursively split into smaller size regions and residualdata in the coding unit may be divided according to the transformationhaving a tree structure according to transformation depths.

According to an exemplary embodiment, a transformation depth indicatingthe number of times splitting is performed to reach the transformationunit by splitting the height and width of the coding unit may also beset in the transformation unit. For example, when the size of atransformation unit of a current coding unit is 2N×2N, a transformationdepth may be set to 0. When the size of a transformation unit is N×N,the transformation depth may be set to 1. In addition, when the size ofthe transformation unit is N/2×N/2, the transformation depth may be setto 2. That is, the transformation unit according to a tree structure mayalso be set according to the transformation depth.

Encoding information according to coding units corresponding to a codeddepth requires not only information about the coded depth, but alsoabout information related to prediction encoding and transformation.Accordingly, the coding unit determiner 120 not only determines a codeddepth having the smallest encoding error, but also determines apartition type in a prediction unit, a prediction mode according toprediction units, and a size of a transformation unit fortransformation.

Coding units and a prediction unit/partition according to a treestructure in a maximum coding unit, and a method of determining atransformation unit, according to exemplary embodiments, will bedescribed in detail later with reference to FIGS. 10 through 20.

The coding unit determiner 120 may measure an encoding error of deepercoding units according to depths by using Rate-Distortion Optimizationbased on Lagrange multipliers.

The output unit 130 outputs the image data of the maximum coding unit,which is encoded based on the at least one coded depth determined by thecoding unit determiner 120, and information about the encoding modeaccording to the coded depth, in bit streams.

The encoded image data may be obtained by encoding residual data of animage.

The information about the encoding mode according to the coded depth mayinclude information about the coded depth, the partition type in theprediction unit, the prediction mode, and the size of the transformationunit.

The information about the coded depth may be defined by using splitinformation according to depths, which indicates whether encoding isperformed on coding units of a lower depth instead of a current depth.If the current depth of the current coding unit is the coded depth,image data in the current coding unit is encoded and output, and thusthe split information may be defined not to split the current codingunit to a lower depth. Alternatively, if the current depth of thecurrent coding unit is not the coded depth, the encoding is performed onthe coding unit of the lower depth, and thus the split information maybe defined to split the current coding unit to obtain the coding unitsof the lower depth.

If the current depth is not the coded depth, encoding is performed onthe coding unit that is split into the coding unit of the lower depth.Because at least one coding unit of the lower depth exists in one codingunit of the current depth, the encoding is repeatedly performed on eachcoding unit of the lower depth, and thus the encoding may be recursivelyperformed for the coding units having the same depth.

Because the coding units having a tree structure are determined for onemaximum coding unit, and information about at least one encoding mode isdetermined for a coding unit of a coded depth, information about atleast one encoding mode may be determined for one maximum coding unit.Also, a coded depth of the image data of the maximum coding unit may bedifferent according to locations because the image data ishierarchically split according to depths, and thus information about thecoded depth and the encoding mode may be set for the image data.

Accordingly, the output unit 130 may assign encoding information about acorresponding coded depth and an encoding mode to at least one of thecoding unit, the prediction unit, and a minimum unit included in themaximum coding unit.

The minimum unit according to an exemplary embodiment is a rectangulardata unit obtained by splitting the minimum coding unit constituting thelowermost depth by 4. Alternatively, the minimum unit may be a maximumrectangular data unit having a maximum size, which is included in all ofthe coding units, prediction units, partition units, and transformationunits included in the maximum coding unit.

For example, the encoding information output through the output unit 130may be classified into encoding information according to coding unitsand encoding information according to prediction units. The encodinginformation according to the coding units may include the informationabout the prediction mode and about the size of the partitions. Theencoding information according to the prediction units may includeinformation about an estimated direction of an inter mode, about areference image index of the inter mode, about a motion vector, about achroma component of an intra mode, and about an interpolation method ofthe intra mode.

Also, information about a maximum size of the coding unit definedaccording to pictures, slices, or GOPs, and information about a maximumdepth may be inserted into a header of a bit stream, an SPS or a PPS.

In addition, information about a maximum size of a transformation unitand information about a minimum size of a transformation, which areacceptable for a current video, may also be output via a header of a bitstream, an SPS or a PPS. The output unit 130 may encode and outputreference information related to prediction, prediction information, andslice segment type information.

In the video encoding apparatus 100, a deeper coding unit may be acoding unit obtained by dividing a height or width of a coding unit ofan upper depth, which is one layer above, by two. In other words, whenthe size of the coding unit of the current depth is 2N×2N, the size ofthe coding unit of the lower depth is N×N. Also, the coding unit of thecurrent depth having a size of 2N×2N may include a maximum value 4 ofthe coding unit of the lower depth.

Accordingly, the video encoding apparatus 100 may form the coding unitshaving a tree structure by determining coding units having an optimumshape and an optimum size for each maximum coding unit, based on thesize of the maximum coding unit and the maximum depth determinedconsidering characteristics of the current picture. Also, becauseencoding may be performed on each maximum coding unit by using any oneof various prediction modes and transformations, an optimum encodingmode may be determined considering characteristics of the coding unit ofvarious image sizes.

Thus, if an image having high resolution or a large amount of data isencoded in a conventional macroblock, the number of macroblocks perpicture excessively increases. Accordingly, the number of pieces ofcompressed information generated for each macroblock increases, and thusit is difficult to transmit the compressed information and datacompression efficiency decreases. However, by using the video encodingapparatus 100, image compression efficiency may be increased because acoding unit is adjusted while considering characteristics of an imagewhile increasing a maximum size of a coding unit while considering asize of the image.

The video encoding apparatus 100 may perform as the video encodingapparatus 10. That is, the coding unit determiner 120 may correspond tothe slice segment encoder 12, and the output unit 130 may correspond tothe slice segment transmitter 14.

Also, the video encoding apparatus 101 may be applied as the videoencoding apparatus 100 according to the present exemplary embodiment.That is, the coding unit determiner 120 may perform operations of thesub-region divider 102 and the sub-region encoder 104.

The coding unit determiner 120 of the exemplary embodiment mayindependently encode each of the tiles in the picture. Also, the codingunit determiner 120 may encode each of at least one slice segment in thepicture. Also, a plurality of maximum coding units included in thecurrent slice segment may be encoded according to the raster scanningorder of the maximum coding units in the tile. Accordingly, the codingunits having a tree structure may be determined in each of the maximumcoding units in each of the slice segments.

Also, a relationship between the slice segment and the tile may satisfyone of the following conditions: (i) the maximum coding units includedin one slice segment may be included in the same tile, (ii) the maximumcoding units included in one tile may be included in the same slicesegment, and (iii) the maximum coding units included in one slicesegment may be included in the same tile, and at the same time, themaximum coding units included in one tile may be included in one sameslice segment.

For example, if condition (i) is satisfied, the at least one slicesegment included in the current tile does not span over the boundary ofthe current tile. That is, each of the slice segments has to becompletely included in the tile.

A relationship between the slice and the tile may satisfy one of thefollowing conditions: (i) the maximum coding units included in one slicemay be included in the same tile, (ii) the maximum coding units includedin one tile may be included in the same slice, and (iii) the maximumcoding units included in one slice may be included in the same tile, andat the same time, the maximum coding units included in one tile may beincluded in one same slice.

The output unit 130 of the present exemplary embodiment may generate aslice segment header including information representing whether thecurrent slice segment is an initial slice segment in the currentpicture.

The output unit 130 of the present exemplary embodiment may addinformation representing whether the current slice segment is adependent slice segment that uses slice header information of theprevious slice segment, if the current slice segment is not the initialslice segment.

The output unit 130 of the present exemplary embodiment may transmit theslice segment header and symbols of the slice segment of each of theslice segments.

FIG. 9 is a block diagram of a video decoding apparatus 200 based oncoding units according to a tree structure, according to an exemplaryembodiment.

The video decoding apparatus 200 based on the coding units according toa tree structure includes a receiver 210, an image data and encodinginformation extractor 220, and an image data decoder 230. Hereinafter,for convenience of description, the video decoding apparatus 200 usingvideo prediction based on coding units according to a tree structurewill be referred to as the ‘video decoding apparatus 200’.

Definitions of various terms and expressions, such as a coding unit, adepth, a prediction unit, a transformation unit, and information aboutvarious encoding modes, for decoding operations of the video decodingapparatus 200 are identical to those described with reference to FIG. 8and the video encoding apparatus 100.

The receiver 210 receives and parses a bit stream of an encoded video.The image data and encoding information extractor 220 extracts encodedimage data for each coding unit from the parsed bit stream, wherein thecoding units have a tree structure according to each maximum codingunit, and outputs the extracted image data to the image data decoder230. The image data and encoding information extractor 220 may extractinformation about a maximum size of a coding unit of a current picture,from a header about the current picture, an SPS, or a PPS.

Also, the image data and encoding information extractor 220 extractsinformation about a coded depth and an encoding mode for the codingunits having a tree structure according to each maximum coding unit,from the parsed bit stream. The extracted information about the codeddepth and the encoding mode is output to the image data decoder 230. Inother words, the image data in a bit stream is split into the maximumcoding units so that the image data decoder 230 decodes the image datafor each maximum coding unit.

The information about the coded depth and the encoding mode according tothe maximum coding unit may be set for information about at least onecoding unit corresponding to the coded depth, and information about anencoding mode may include information about a partition type of acorresponding coding unit corresponding to the coded depth, about aprediction mode, and a size of a transformation unit. Also, splittinginformation according to depths may be extracted as the informationabout the coded depth.

The information about the coded depth and the encoding mode according toeach maximum coding unit extracted by the image data and encodinginformation extractor 220 is information about a coded depth and anencoding mode determined to generate a minimum encoding error when anencoder, such as the video encoding apparatus 100, repeatedly performsencoding for each deeper coding unit according to depths according toeach maximum coding unit. Accordingly, the video decoding apparatus 200may restore an image by decoding the image data according to a codeddepth and an encoding mode that generates the minimum encoding error.

Because encoding information about the coded depth and the encoding modemay be assigned to a predetermined data unit from among a correspondingcoding unit, a prediction unit, and a minimum unit, the image data andencoding information extractor 220 may extract the information about thecoded depth and the encoding mode according to the predetermined dataunits. The predetermined data units to which the same information aboutthe coded depth and the encoding mode is assigned may be determined tobe the data units included in the same maximum coding unit.

The image data decoder 230 restores the current picture by decoding theimage data in each maximum coding unit based on the information aboutthe coded depth and the encoding mode according to the maximum codingunits. In other words, the image data decoder 230 may decode the encodedimage data based on the extracted information about the partition type,the prediction mode, and the transformation unit for each coding unitfrom among the coding units having a tree structure included in eachmaximum coding unit. A decoding process may include a prediction processincluding intra prediction and motion compensation, and inversetransformation.

The image data decoder 230 may perform intra prediction or motioncompensation according to partitions and a prediction mode of eachcoding unit, based on the information about the partition type and theprediction mode of the prediction unit of the coding unit according tocoded depths.

In addition, the image data decoder 230 may read transformation unitinformation according to a tree structure for each coding unit todetermine transformation units for each coding unit and perform inversetransformation based on transformation units on each coding unit, forinverse transformation of each maximum coding unit. Via the inversetransformation, a pixel value of a spatial region of the coding unit maybe restored.

The image data decoder 230 may determine at least one coded depth of acurrent maximum coding unit by using split information according todepths. If the split information indicates that image data is no longersplit in the current depth, the current depth is a coded depth.Accordingly, the image data decoder 230 may decode encoded data of atleast one coding unit corresponding to each coded depth in the currentmaximum coding unit by using the information about the partition type ofthe prediction unit, the prediction mode, and the size of thetransformation unit.

In other words, data units containing the encoding information includingthe same split information may be collected by observing the encodinginformation set assigned for the predetermined data unit from among thecoding unit, the prediction unit, and the minimum unit, and thecollected data units may be considered to be one data unit to be decodedby the image data decoder 230 in the same encoding mode. For each codingunit determined as described above, information about an encoding modemay be obtained to decode the current coding unit.

The receiver 210 may correspond to the sub-region receiver 102 of thevideo decoding apparatus 202 described with reference to FIG. 2C. Theimage data decoder 230 may correspond to the sub-region decoder 204 ofthe video decoding apparatus 202 described with reference to FIG. 2C.

The receiver 210 may correspond to the slice segment parser 22 of thevideo decoding apparatus 20 described with reference to FIG. 2C. Theimage data decoder 230 may correspond to the slice segment decoder 24 ofthe video decoding apparatus 20 described with reference to FIG. 2C.

The receiver 210 of the present exemplary embodiment may receive a bitstream generated by encoding the picture by the tile and the slicesegment units. Also, the bit stream for each of the slice segments mayinclude a slice segment header and encoding symbols of the slicesegments.

The receiver 210 may parse information representing whether the currentslice segment is an initial slice segment in the current picture fromthe slice segment header of the current slice segment. When it isdetermined that the current slice segment is not the initial slicesegment from the parsed information, the receiver 210 may further parseinformation representing whether the current slice segment is adependent slice segment using the slice header information of theprevious slice segment, from the current slice segment header.

When it is determined that the current slice segment is the initialslice segment from the parsed information, the receiver 210 does notparse the information representing whether the current slice segment isa dependent slice segment, from the current slice segment header. If thecurrent slice segment is the initial slice segment, the receiver 210 mayparse the information representing whether the current slice segment isthe initial slice segment of an initial slice segment header of thepicture and default information about the current slice segment from aninitial slice segment header of the picture.

When it is determined that the current slice segment is the dependentslice segment from the information parsed from the current slice segmentheader, the receiver 210 may determine various pieces of headerinformation parsed from the header of the previous slice segment as thedefault information of the current slice segment.

The image data decoder 230 of the present exemplary embodiment maydecode the current slice segment by using the information parsed fromthe slice segment header and symbols of the current slice segment.

Also, the image data decoder 230 of the present exemplary embodiment mayreconstruct the decoded current slice segment, and may reconstruct thepicture by combining the reconstructed slice segments.

Also, the image data decoder 230 may reconstruct the picture bycombining the slice segments that are decoded for each of the tiles.

The receiver 210 may parse symbols of a plurality of maximum codingunits included in the current slice segment according to a scanningorder in the tile, with respect to each of the slice segments. Also, theimage data decoder 230 of the present exemplary embodiment may decodethe maximum coding units according to the raster scanning order by usingthe parsed symbols of the maximum coding units.

The maximum coding units that are encoded (decoded) according to thescanning order in each of the tiles and in each of the slice segmentssatisfy one of the following conditions: (i) the maximum coding unitsincluded in one slice segment may be included in the same tile; (ii) themaximum coding units included in one tile may be included in the sameslice segment; and (iii) the maximum coding units included in one slicesegment may be included in the same tile, and the maximum coding unitsincluded in one tile may be included in the same slice segment.

For example, if condition (i) is satisfied, the slice segment includedin the current tile may be decoded so as not to span over the boundaryof the current tile.

The maximum coding units that are encoded (decoded) according to thescanning order in each of the tiles and in each of the slices satisfyone of the following conditions: (i) the maximum coding units includedin one slice may be included in the same tile; (ii) the maximum codingunits included in one tile may be included in the same slice; and (iii)the maximum coding units included in one slice may be included in thesame tile, and the maximum coding units included in one tile may beincluded in the same slice.

Therefore, the image data decoder 230 of the present exemplaryembodiment sequentially decodes the maximum coding units in each of theslice segments to reconstruct the slice segment, and reconstructs thetiles to reconstruct the picture consisting of the tiles.

Also, the image data decoder 230 of the present exemplary embodiment mayreconstruct each of the tiles by sequentially decoding the maximumcoding units in each of the tiles, and may reconstruct the pictureconsisting of the reconstructed tiles.

That is, the video decoding apparatus 200 may obtain information aboutat least one coding unit that generates the minimum encoding error whenencoding is recursively performed for each maximum coding unit, and mayuse the information to decode the current picture. In other words, thecoding units, which have a tree structure, determined to be the optimumcoding units in each maximum coding unit may be decoded.

Accordingly, even if image data has high resolution and is a largeamount of data, the image data may be efficiently decoded and restoredby using a size of a coding unit and an encoding mode, which areadaptively determined according to characteristics of the image data, byusing information about an optimum encoding mode received from anencoder.

FIG. 10 is a diagram for describing a concept of coding units accordingto an exemplary embodiment.

A size of a coding unit may be expressed in width×height, and may be64×64, 32×32, 16×16, and 8×8. A coding unit of 64×64 may be split intopartitions of 64×64, 64×32, 32×64, or 32×32, and a coding unit of 32×32may be split into partitions of 32×32, 32×16, 16×32, or 16×16, a codingunit of 16×16 may be split into partitions of 16×16, 16×8, 8×16, or 8×8,and a coding unit of 8×8 may be split into partitions of 8×8, 8×4, 4×8,or 4×4.

In video data 310, a resolution is 1920×1080, a maximum size of a codingunit is 64, and a maximum depth is 2. In video data 320, a resolution is1920×1080, a maximum size of a coding unit is 64, and a maximum depth is3. In video data 330, a resolution is 352×288, a maximum size of acoding unit is 16, and a maximum depth is 1. The maximum depth shown inFIG. 10 denotes a total number of splits from a maximum coding unit to aminimum coding unit.

If a resolution is high or an amount of data is large, a maximum size ofa coding unit may be large to not only increase encoding efficiency butalso to accurately reflect characteristics of an image. Accordingly, themaximum size of the coding unit of the video data 310 and 320 having ahigher resolution than the video data 330 may be 64.

Because the maximum depth of the video data 310 is 2, coding units 315of the video data 310 may include maximum coding units having a longaxis size of 64 and coding units having long axis sizes of 32 and 16because depths deepen to two layers by splitting the maximum coding unittwice. Because the maximum depth of the video data 330 is 1, codingunits 335 of the video data 330 may include maximum coding units havinga long axis size of 16 and coding units having a long axis size of 8because depths deepen to one layer by splitting the maximum coding unitonce.

Because the maximum depth of the video data 320 is 3, coding units 325of the video data 320 may include a maximum coding unit having a longaxis size of 64 and coding units having long axis sizes of 32, 16, and 8because the depths deepen to 3 layers by splitting the maximum codingunit three times. As a depth deepens, detailed information may beprecisely expressed.

FIG. 11 is a block diagram of an image encoder 400 based on codingunits, according to an exemplary embodiment.

The image encoder 400 performs operations of the coding unit determiner120 of the video encoding apparatus 100 to encode image data. In otherwords, an intra predictor 410 performs intra prediction on coding unitsin an intra mode, from a current frame 405, and a motion estimator 420and a motion compensator 425 perform inter prediction and motioncompensation on coding units in an inter mode from the current frame 405by using the current frame 405, and a reference frame 495.

Data output from the intra predictor 410, the motion estimator 420, andthe motion compensator 425 is output as a quantized transformationcoefficient through a transformer 430 and a quantizer 440. The quantizedtransformation coefficient is restored as data in a spatial domainthrough an inverse quantizer 460 and an inverse transformer 470, and therestored data in the spatial domain is output as the reference frame 495after being post-processed through a deblocking unit 480 and an SAOfilter 490. The quantized transformation coefficient may be output as abit stream 455 through an entropy encoder 450.

In order for the image encoder 400 to be applied in the video encodingapparatus 100, all elements of the image encoder 400, i.e., the intrapredictor 410, the motion estimator 420, the motion compensator 425, thetransformer 430, the quantizer 440, the entropy encoder 450, the inversequantizer 460, the inverse transformer 470, the deblocking unit 480, andthe SAO filter 490, perform operations based on each coding unit fromamong coding units having a tree structure while considering the maximumdepth of each maximum coding unit.

Specifically, the intra predictor 410, the motion estimator 420, and themotion compensator 425 determine partitions and a prediction mode ofeach coding unit from among the coding units having a tree structurewhile considering the maximum size and the maximum depth of a currentmaximum coding unit, and the transformer 430 determines the size of thetransformation unit in each coding unit from among the coding unitshaving a tree structure.

The image encoder 400 may perform the encoding operation on each of themaximum coding units, according to the characteristics of the slicesegments, the tiles, and the slices described with reference to FIGS. 1Athrough 7. In particular, the entropy encoder 450 may correspond to theslice segment transmitter 14 according to the exemplary embodiment.

FIG. 12 is a block diagram of an image decoder 500 based on codingunits, according to an exemplary embodiment.

A parser 510 parses encoded image data to be decoded and informationabout encoding required for decoding, from a bit stream 505. The encodedimage data is output as inverse quantized data through an entropydecoder 520 and an inverse quantizer 530, and the inverse quantized datais restored to image data in a spatial domain through an inversetransformer 540.

An intra predictor 550 performs intra prediction on coding units in anintra mode with respect to the image data in the spatial domain, and amotion compensator 560 performs motion compensation on coding units inan inter mode by using a reference frame 585.

The image data in the spatial domain, which passed through the intrapredictor 550 and the motion compensator 560, may be output as arestored frame 595 after being post-processed through a deblocking unit570 and an SAO filter 580. Also, the image data that is post-processedthrough the deblocking unit 570 and the SAO filter 580 may be output asthe reference frame 585.

In order to decode the image data in the image data decoder 230 of thevideo decoding apparatus 200, the image decoder 500 may performoperations that are performed after the parser 510 performs anoperation.

In order for the image decoder 500 to be applied in the video decodingapparatus 200, all elements of the image decoder 500, i.e., the parser510, the entropy decoder 520, the inverse quantizer 530, the inversetransformer 540, the intra predictor 550, the motion compensator 560,the deblocking unit 570, and the SAO filter 580, perform operationsbased on coding units having a tree structure for each maximum codingunit.

Specifically, the intra prediction 550 and the motion compensator 560perform operations based on partitions and a prediction mode for each ofthe coding units having a tree structure, and the inverse transformer540 performs operations based on a size of a transformation unit foreach coding unit. The image decoder 500 may perform the decodingoperation on each of the maximum coding units, according to thecharacteristics of the slice segments, the tiles, and the slicesdescribed with reference to FIGS. 1A through 7. In particular, theentropy decoder 520 may correspond to the slice segment parser 22according to the exemplary embodiment.

FIG. 13 is a diagram illustrating deeper coding units according todepths, and partitions, according to an exemplary embodiment.

The video encoding apparatus 100 and the video decoding apparatus 200use hierarchical coding units so as to consider characteristics of animage. A maximum height, a maximum width, and a maximum depth of codingunits may be adaptively determined according to the characteristics ofthe image, or may be differently set by a user. Sizes of deeper codingunits according to depths may be determined according to thepredetermined maximum size of the coding unit.

In a hierarchical structure 600 of coding units, according to anexemplary embodiment, the maximum height and the maximum width of thecoding units are each 64, and the maximum depth is 3. In this case, themaximum depth refers to a total number of times the coding unit is splitfrom the maximum coding unit to the minimum coding unit. Because a depthdeepens along a vertical axis of the hierarchical structure 600, aheight and a width of the deeper coding unit are each split. Also, aprediction unit and partitions, which are bases for prediction encodingof each deeper coding unit, are shown along a horizontal axis of thehierarchical structure 600.

In other words, a coding unit 610 is a maximum coding unit in thehierarchical structure 600, wherein a depth is 0 and a size, i.e., aheight by width, is 64×64. The depth deepens along the vertical axis ofthe hierarchical structure 600, and a coding unit 620 having a size of32×32 and a depth of 1, a coding unit 630 having a size of 16×16 and adepth of 2, and a coding unit 640 having a size of 8×8 and a depth of 3.The coding unit 640 having a size of 8×8 and a depth of 4 is a minimumcoding unit (smallest coding unit, SCU).

The prediction unit and the partitions of a coding unit are arrangedalong the horizontal axis according to each depth. In other words, ifthe coding unit 610 having a size of 64×64 and a depth of 0 is aprediction unit, the prediction unit may be split into partitionsincluded in the coding unit 610, i.e. a partition having a size of64×64, partitions 612 having a size of 64×32, partitions 614 having asize of 32×64, or partitions 616 having a size of 32×32.

Similarly, a prediction unit of the coding unit 620 having a size of32×32 and a depth of 1 may be split into partitions included in thecoding unit 620, i.e. a partition having a size of 32×32, partitions 622having a size of 32×16, partitions 624 having a size of 16×32, andpartitions 626 having a size of 16×16.

Similarly, a prediction unit of the coding unit 630 having a size of16×16 and a depth of 2 may be split into partitions included in thecoding unit 630, i.e. a partition having a size of 16×16 included in thecoding unit 630, partitions 632 having a size of 16×8, partitions 634having a size of 8×16, and partitions 636 having a size of 8×8.

Similarly, a prediction unit of the coding unit 640 having a size of 8×8and a depth of 3 may be split into partitions included in the codingunit 640, i.e. a partition having a size of 8×8 included in the codingunit 640, partitions 642 having a size of 8×4, partitions 644 having asize of 4×8, and partitions 646 having a size of 4×4.

In order to determine the at least one coded depth of the coding unitsconstituting the maximum coding unit 610, the coding unit determiner 120of the video encoding apparatus 100 performs encoding on coding unitscorresponding to each depth included in the maximum coding unit 610.

The number of deeper coding units according to depths including data inthe same range and the same size increases as the depth deepens. Forexample, four coding units corresponding to a depth of 2 are required tocover data that is included in one coding unit corresponding to a depthof 1. Accordingly, in order to compare encoding results of the same dataaccording to depths, a coding unit corresponding to a depth of 1 andfour coding units corresponding to a depth of 2 are each encoded.

In order to perform encoding for a current depth from among the depths,the least encoding error may be selected for the current depth byperforming encoding on each prediction unit in the coding unitscorresponding to the current depth, along the horizontal axis of thehierarchical structure 600. Alternatively, the minimum encoding errormay be searched for by comparing the smallest encoding errors accordingto depths, by performing encoding for each depth as the depth deepensalong the vertical axis of the hierarchical structure 600. A depth and apartition having the minimum encoding error in the coding unit 610 maybe selected as the coded depth and a partition type of the coding unit610.

FIG. 14 is a diagram for describing a relationship between a coding unit710 and transformation units 720, according to an exemplary embodiment.

The video encoding apparatus 100 or the video decoding apparatus 200according to the exemplary embodiments encodes or decodes an imageaccording to coding units having sizes smaller than or equal to amaximum coding unit for each maximum coding unit. Sizes oftransformation units for transformation during encoding may be selectedbased on data units that are not larger than a corresponding codingunit.

For example, in the video encoding apparatus 100 or the video decodingapparatus 200, if a size of the coding unit 710 is 64×64, transformationmay be performed by using the transformation units 720 having a size of32×32.

Also, data of the coding unit 710 having a size of 64×64 may be encodedby performing the transformation on each of the transformation unitshaving a size of 32×32, 16×16, 8×8, and 4×4, which are smaller than64×64, and then a transformation unit having the smallest coding errormay be selected.

FIG. 15 is a diagram for describing encoding information of coding unitscorresponding to a coded depth, according to an exemplary embodiment.

The output unit 130 of the video encoding apparatus 100 may encode andtransmit information 800 about a partition type, information 810 about aprediction mode, and information 820 about a size of a transformationunit for each coding unit corresponding to a coded depth, as informationabout an encoding mode.

The information 800 about the partition type indicates information abouta shape of a partition obtained by splitting a prediction unit of acurrent coding unit, wherein the partition is a data unit for predictionencoding the current coding unit. For example, a current coding unitCU_0 having a size of 2N×2N may be split into any one of a partition 802having a size of 2N×2N, a partition 804 having a size of 2N×N, apartition 806 having a size of N×2N, and a partition 808 having a sizeof N×N. Here, the information 800 about a partition type is set toindicate one of the partition 804 having a size of 2N×N, the partition806 having a size of N×2N, and the partition 808 having a size of N×N.

The information 810 indicates a prediction mode of each partition. Forexample, the information 810 may indicate a mode of prediction encodingperformed on a partition indicated by the information 800, i.e., anintra mode 812, an inter mode 814, or a skip mode 816.

The information 820 indicates a transformation unit to be based on whentransformation is performed on a current coding unit. For example, thetransformation unit may be a first intra transformation unit 822, asecond intra transformation unit 824, a first inter transformation unit826, or a second inter transformation unit 828.

The image data and encoding information extractor 210 of the videodecoding apparatus 200 may extract and use the information 800, 810, and820 for decoding, according to each deeper coding unit.

FIG. 16 is a diagram of deeper coding units according to depths,according to an exemplary embodiment.

Split information may be used to indicate a change of a depth. The splitinformation indicates whether a coding unit of a current depth is splitinto coding units of a lower depth.

A prediction unit 910 for prediction encoding a coding unit 900 having adepth of 0 and a size of 2N_(—)0×2N_(—)0 may include partitions of apartition type 912 having a size of 2N_(—)0×2N_(—)0, a partition type914 having a size of 2N_(—)0×N_(—)0, a partition type 916 having a sizeof N_(—)0×2N_(—)0, and a partition type 918 having a size ofN_(—)0×N_(—)0. FIG. 16 only illustrates the partition types 912 through918 which are obtained by symmetrically splitting the prediction unit910, but a partition type is not limited thereto, and the partitions ofthe prediction unit 910 may include asymmetrical partitions, partitionshaving a predetermined shape, and partitions having a geometrical shape.

Prediction encoding is repeatedly performed on one partition having asize of 2N_(—)0×2N_(—)0, two partitions having a size of 2N_(—)0×N_(—)0,two partitions having a size of N_(—)0×2N_(—)0, and four partitionshaving a size of N_(—)0×N_(—)0, according to each partition type. Theprediction encoding in an intra mode and an inter mode may be performedon the partitions having sizes of 2N_(—)0×2N_(—)0, N_(—)0×2N_(—)0,2N_(—)0×N_(—)0, and N_(—)0×N_(—)0. The prediction encoding in a skipmode is performed only on the partition having a size of2N_(—)0×2N_(—)0.

If an encoding error is smallest in one of the partition types 912through 918 having sizes of 2N_(—)0×2N_(—)0, N_(—)0×2N_(—)0,2N_(—)0×N_(—)0, and N_(—)0×N_(—)0, the prediction unit 910 may not besplit into a lower depth.

If the encoding error is the smallest in the partition type 918 having asize of N_(—)0×N_(—)0, a depth is changed from 0 to 1 to split thepartition type 918 in operation 920, and encoding is repeatedlyperformed on coding units 930 having a depth of 2 and a size ofN_(—)0×N_(—)0, to search for a minimum encoding error.

A prediction unit 940 for prediction encoding the coding unit 930 havinga depth of 1 and a size of 2N_(—)1×2N_(—)1 (=N_(—)0×N_(—)0) may includepartitions of a partition type 942 having a size of 2N_(—)1×2N_(—)1, apartition type 944 having a size of 2N_(—)1×N_(—)1, a partition type 946having a size of N_(—)1×2N_(—)1, and a partition type 948 having a sizeof N_(—)1×N_(—)1.

If an encoding error is the smallest in the partition type 948, a depthis changed from 1 to 2 to split the partition type 948 in operation 950,and encoding is repeatedly performed on coding units 960, which have adepth of 2 and a size of N_(—)2×N_(—)2, to search for a minimum encodingerror.

When a maximum depth is d, split operation according to each depth maybe performed up to when a depth becomes d−1, and split information maybe encoded up to when a depth is one of 0 to d−2. In other words, whenencoding is performed up to when the depth is d−1 after a coding unitcorresponding to a depth of d−2 is split in operation 970, a predictionunit 990 for prediction encoding a coding unit 980 having a depth of d−1and a size of 2N_(d−1)×2N_(d−1) may include partitions of a partitiontype 992 having a size of 2N_(d−1)×2N_(d−1), a partition type 994 havinga size of 2N_(d−1)×N_(d−1), a partition type 996 having a size ofN_(d−1)×2N_(d−1), and a partition type 998 having a size ofN_(d−1)×N_(d−1).

Prediction encoding may be repeatedly performed on one partition havinga size of 2N_(d−1)×2N_(d−1), two partitions having a size of2N_(d−1)×N_(d−1), two partitions having a size of N_(d−1)×2N_(d−1), fourpartitions having a size of N_(d−1)×N_(d−1) from among the partitiontypes 992 through 998 to search for a partition type having a minimumencoding error.

Even when the partition type 998 has the minimum encoding error, becausea maximum depth is d−1, a coding unit CU_(d−1) having a depth of d−1 isno longer split to a lower depth, and a coded depth for the coding unitsconstituting the current maximum coding unit 900 is determined to be d−1and a partition type may be determined to be N_(d−1)×N_(d−1). Also,because the maximum depth is d−1, split information for the minimumcoding unit 980 is not set.

A data unit 999 may be a ‘minimum unit’ for the current maximum codingunit. A minimum unit according to an exemplary embodiment may be arectangular data unit obtained by splitting a minimum coding unit by 4.By performing the encoding repeatedly, the video encoding apparatus 100may select a depth having the least encoding error by comparing encodingerrors according to depths of the coding unit 900 to determine a codeddepth, and set a corresponding partition type and a prediction mode asan encoding mode of the coded depth.

As such, the minimum encoding errors according to depths are compared inall of the depths of 0 through d, and a depth having the least encodingerror may be determined as a coded depth. The coded depth, the partitiontype of the prediction unit, and the prediction mode may be encoded andtransmitted as information about an encoding mode. Also, because acoding unit is split from a depth of 0 to a coded depth, only splitinformation of the coded depth is set to 0, and split information ofdepths excluding the coded depth is set to 1.

The image data and encoding information extractor 220 of the videodecoding apparatus 200 may extract and use the information about thecoded depth and the prediction unit of the coding unit 900 to decode thepartition 912. The video decoding apparatus 200 may determine a depth,in which split information is 0, as a coded depth by using splitinformation according to depths, and use information about an encodingmode of the corresponding depth for decoding.

FIGS. 17 through 19 are diagrams for describing a relationship betweencoding units 1010, prediction units 1060, and transformation units 1070,according to an exemplary embodiment.

The coding units 1010 are coding units having a tree structure,corresponding to coded depths determined by the video encoding apparatus100, in a maximum coding unit. The prediction units 1060 are partitionsof prediction units of each of the coding units 1010, and thetransformation units 1070 are transformation units of each of the codingunits 1010.

When a depth of a maximum coding unit is 0 in the coding units 1010,depths of coding units 1012 and 1054 are 1, depths of coding units 1014,1016, 1018, 1028, 1050, and 1052 are 2, depths of coding units 1020,1022, 1024, 1026, 1030, 1032, and 1048 are 3, and depths of coding units1040, 1042, 1044, and 1046 are 4.

In the prediction units 1060, some encoding units 1014, 1016, 1022,1032, 1048, 1050, 1052, and 1054 are obtained by splitting the codingunits. In other words, partition types in the coding units 1014, 1022,1050, and 1054 have a size of 2N×N, partition types in the coding units1016, 1048, and 1052 have a size of N×2N, and a partition type of thecoding unit 1032 has a size of N×N. Prediction units and partitions ofthe coding units 1010 are smaller than or equal to each coding unit.

Transformation or inverse transformation is performed on image data ofthe coding unit 1052 in the transformation units 1070 in a data unitthat is smaller than the coding unit 1052. Also, the coding units 1014,1016, 1022, 1032, 1048, 1050, 1052, and 1054 in the transformation units1070 are different from those in the prediction units 1060 in terms ofsizes and shapes. In other words, the video encoding and decodingapparatuses 100 and 200 may perform intra prediction, motion prediction,motion compensation, transformation, and inverse transformationindividually on a data unit in the same coding unit.

Accordingly, encoding is recursively performed on each of the codingunits having a hierarchical structure in each region of a maximum codingunit to determine an optimum coding unit, and thus coding units having arecursive tree structure may be obtained. Encoding information mayinclude split information about a coding unit, information about apartition type, information about a prediction mode, and informationabout a size of a transformation unit. Table 1 shows the encodinginformation that may be set by the video encoding and decodingapparatuses 100 and 200.

TABLE 1 Split Information 0 (Encoding on Coding Unit having Size of 2N ×2N and Current Depth of d) Prediction Split Mode Partition Type Size ofTransformation Unit Information 1 Intra Symmet- Asymmet- Split SplitRepeatedly Inter rical Parti- rical Parti- Informa- Informa- Encode Skiption Type tion Type tion 0 of tion 1 of Coding (Only Transfor- Transfor-Units 2N × 2N) mation Unit mation Unit having 2N × 2N 2N × nU 2N × 2N N× N (Symmet- Lower 2N × N 2N × nD rical Type) Depth of N × 2N nL × 2NN/2 × N/2 d + 1 N × N nR × 2N (Asymmet- rical Type)

The output unit 130 of the video encoding apparatus 100 may output theencoding information about the coding units having a tree structure, andthe image data and encoding information extractor 220 of the videodecoding apparatus 200 may extract the encoding information about thecoding units having a tree structure from a received bit stream.

Split information indicates whether a current coding unit is split intocoding units of a lower depth. If split information of a current depth dis 0, a depth, in which a current coding unit is no longer split into alower depth, is a coded depth, and thus information about a partitiontype, a prediction mode, and a size of a transformation unit may bedefined for the coded depth. If the current coding unit is further splitaccording to the split information, encoding is independently performedon four split coding units of a lower depth.

A prediction mode may be one of an intra mode, an inter mode, and a skipmode. The intra mode and the inter mode may be defined in all partitiontypes, and the skip mode is defined only in a partition type having asize of 2N×2N.

The information about the partition type may indicate symmetricalpartition types having sizes of 2N×2N, 2N×N, N×2N, and N×N, which areobtained by symmetrically splitting a height or a width of a predictionunit, and asymmetrical partition types having sizes of 2N×nU, 2N×nD,nL×2N, and nR×2N, which are obtained by asymmetrically splitting theheight or width of the prediction unit. The asymmetrical partition typeshaving sizes of 2N×nU and 2N×nD may be respectively obtained bysplitting the height of the prediction unit in 1:3 and 3:1, and theasymmetrical partition types having sizes of nL×2N and nR×2N may berespectively obtained by splitting the width of the prediction unit in1:3 and 3:1.

The size of the transformation unit may be set to be two types in theintra mode and two types in the inter mode. In other words, if splitinformation of the transformation unit is 0, the size of thetransformation unit may be 2N×2N, which is the size of the currentcoding unit. If split information of the transformation unit is 1, thetransformation units may be obtained by splitting the current codingunit. Also, if a partition type of the current coding unit having a sizeof 2N×2N is a symmetrical partition type, a size of a transformationunit may be N×N, and if the partition type of the current coding unit isan asymmetrical partition type, the size of the transformation unit maybe N/2×N/2.

The encoding information about coding units having a tree structure mayinclude at least one of a coding unit corresponding to a coded depth, aprediction unit, and a minimum unit. The coding unit corresponding tothe coded depth may include at least one of a prediction unit and aminimum unit containing the same encoding information.

Accordingly, it is determined whether adjacent data units are includedin the same coding unit corresponding to the coded depth by comparingencoding information of the adjacent data units. Also, a correspondingcoding unit corresponding to a coded depth is determined by usingencoding information of a data unit, and thus a distribution of codeddepths in a maximum coding unit may be determined.

Accordingly, if a current coding unit is predicted based on encodinginformation of adjacent data units, encoding information of data unitsin deeper coding units adjacent to the current coding unit may bedirectly referred to and used.

Alternatively, if a current coding unit is predicted based on encodinginformation of adjacent data units, data units adjacent to the currentcoding unit are searched for using encoded information of the dataunits, and the adjacent coding units may be referred to for predictingthe current coding unit.

FIG. 20 is a diagram for describing a relationship between a codingunit, a prediction unit or a partition, and a transformation unit,according to encoding mode information of Table 1.

A maximum coding unit 1300 includes coding units 1302, 1304, 1306, 1312,1314, 1316, and 1318 of coded depths. Here, because the coding unit 1318is a coding unit of a coded depth, split information may be set to 0.Information about a partition type of the coding unit 1318 having a sizeof 2N×2N may be set to be one of a partition type 1322 having a size of2N×2N, a partition type 1324 having a size of 2N×N, a partition type1326 having a size of N×2N, a partition type 1328 having a size of N×N,a partition type 1332 having a size of 2N×nU, a partition type 1334having a size of 2N×nD, a partition type 1336 having a size of nL×2N,and a partition type 1338 having a size of nR×2N.

Split information, for example a transformation unit (TU) size flag, ofa transformation unit is a type of a transformation index. The size ofthe transformation unit corresponding to the transformation index may bechanged according to a prediction unit type or partition type of thecoding unit.

For example, when the partition type is set to be symmetrical, i.e. thepartition type 1322, 1324, 1326, or 1328, a transformation unit 1342having a size of 2N×2N is set if split information (TU size flag) of atransformation unit is 0, and a transformation unit 1344 having a sizeof N×N is set if a TU size flag is 1.

When the partition type is set to be asymmetrical, i.e., the partitiontype 1332, 1334, 1336, or 1338, a transformation unit 1352 having a sizeof 2N×2N is set if a TU size flag is 0, and a transformation unit 1354having a size of N/2×N/2 is set if a TU size flag is 1.

Referring to FIG. 20, the TU size flag is a flag having a value of 0 or1, but the TU size flag is not limited to 1 bit, and a transformationunit may be hierarchically split having a tree structure while the TUsize flag increases from 0. Split information (TU size flag) of atransformation unit may be an example of a transformation index.

In this case, the size of a transformation unit that has been actuallyused may be expressed by using a TU size flag of a transformation unit,according to an exemplary embodiment, together with a maximum size and aminimum size of the transformation unit. According to an exemplaryembodiment, the video encoding apparatus 100 is capable of encodingmaximum transformation unit size information, minimum transformationunit size information, and a maximum TU size flag. The result ofencoding the maximum transformation unit size information, the minimumtransformation unit size information, and the maximum TU size flag maybe inserted into an SPS. According to an exemplary embodiment, the videodecoding apparatus 200 may decode video by using the maximumtransformation unit size information, the minimum transformation unitsize information, and the maximum TU size flag.

For example, (a) if the size of a current coding unit is 64×64 and amaximum transformation unit size is 32×32, (a−1) then the size of atransformation unit may be 32×32 when a TU size flag is 0, (a−2) may be16×16 when the TU size flag is 1, and (a−3) may be 8×8 when the TU sizeflag is 2.

As another example, (b) if the size of the current coding unit is 32×32and a minimum transformation unit size is 32×32, (b−1) then the size ofthe transformation unit may be 32×32 when the TU size flag is 0. Here,the TU size flag cannot be set to a value other than 0, because the sizeof the transformation unit cannot be less than 32×32.

As another example, (c) if the size of the current coding unit is 64×64and a maximum TU size flag is 1, then the TU size flag may be 0 or 1.Here, the TU size flag cannot be set to a value other than 0 or 1.

Thus, if it is defined that the maximum TU size flag is‘MaxTransformSizeIndex’, a minimum transformation unit size is‘MinTransformSize’, and a transformation unit size is ‘RootTuSize’ whenthe TU size flag is 0, then a current minimum transformation unit size‘CurrMinTuSize’ that can be determined in a current coding unit may bedefined by Equation (1):

CurrMinTuSize=max(MinTransformSize,RootTuSize/(2̂MaxTransformSizeIndex))  (1)

Compared to the current minimum transformation unit size ‘CurrMinTuSize’that can be determined in the current coding unit, a transformation unitsize ‘RootTuSize’ when the TU size flag is 0 may denote a maximumtransformation unit size that can be selected in the system. In Equation(1), ‘RootTuSize/(2̂MaxTransformSizeIndex)’ denotes a transformation unitsize when the transformation unit size ‘RootTuSize’, when the TU sizeflag is 0, is split a number of times corresponding to the maximum TUsize flag, and ‘MinTransformSize’ denotes a minimum transformation size.Thus, a smaller value from among ‘RootTuSize/(2̂MaxTransformSizeIndex)’and ‘MinTransformSize’ may be the current minimum transformation unitsize ‘CurrMinTuSize’ that can be determined in the current coding unit.

According to an exemplary embodiment, the maximum transformation unitsize RootTuSize may vary according to the type of a prediction mode.

For example, if a current prediction mode is an inter mode, then‘RootTuSize’ may be determined by using Equation (2) below. In Equation(2), ‘MaxTransformSize’ denotes a maximum transformation unit size and‘PUSize’ denotes a current prediction unit size.

RootTuSize=min(MaxTransformSize,PUSize)  (2)

That is, if the current prediction mode is an inter mode, thetransformation unit size ‘RootTuSize’, when the TU size flag is 0, maybe a smaller value of the maximum transformation unit size and thecurrent prediction unit size.

If a prediction mode of a current partition unit is an intra mode,‘RootTuSize’ may be determined by using Equation (3) below. In Equation(3), ‘PartitionSize’ denotes the size of the current partition unit.

RootTuSize=min(MaxTransformSize,PartitionSize)  (3)

That is, if the current prediction mode is an intra mode, thetransformation unit size ‘RootTuSize’ when the TU size flag is 0 may bea smaller value of the maximum transformation unit size and the size ofthe current partition unit.

However, the current maximum transformation unit size ‘RootTuSize’ thatvaries according to the type of a prediction mode in a partition unit isjust an example and is not limited thereto.

According to the video encoding method based on coding units having atree structure, as described with reference to FIGS. 8 through 20, imagedata of a spatial region is encoded for each coding unit of a treestructure. According to the video decoding method based on coding unitshaving a tree structure, decoding is performed for each maximum codingunit to restore image data of a spatial region. Thus, a picture and avideo that is a picture sequence may be restored. The restored video maybe reproduced by a reproducing apparatus, stored in a storage medium, ortransmitted through a network.

The exemplary embodiments may be written as computer programs and may beimplemented in general-use digital computers that execute the programsusing a computer-readable recording medium. Examples of thecomputer-readable recording medium include magnetic storage media (e.g.,ROM, floppy disks, hard disks, etc.) and optical recording media (e.g.,CD-ROMs or DVDs).

For convenience of description, a video encoding method, including theentropy encoding method described with reference to FIGS. 1A through 20,will be collectively referred to as a ‘video encoding method accordingto the present invention’. In addition, the video decoding method,including the entropy decoding method described with reference to FIGS.1A through 20, will be referred to as a ‘video decoding method accordingto the present invention’.

A video encoding apparatus, including the video encoding apparatus 10,the video encoding apparatus 101, the video encoding apparatus 100, orthe image encoder 400 described with reference to FIGS. 1A through 20,will be referred to as a ‘video encoding apparatus’. In addition, avideo decoding apparatus, including the video decoding apparatus 20, thevideo decoding apparatus 201, the video decoding apparatus 200, or theimage decoder 500 described with reference to FIGS. 1A through 20, willbe referred to as a ‘video decoding apparatus’.

A computer-readable recording medium storing a program, e.g., a disc26000, according to an exemplary embodiment will now be described indetail.

FIG. 21 illustrates a physical structure of a disc 26000 that stores aprogram, according to an exemplary embodiment. The disc 26000, which isa storage medium, may be a hard drive disc, a CD-ROM disc, a Blu-raydisc, or a DVD. The disc 26000 includes a plurality of concentric tracksTr that are each divided into a specific number of sectors Se in acircumferential direction of the disc 26000. In a specific region of thedisc 26000, a program that executes a method of predicting multi-viewvideo, a method of prediction restoring multi-view video, a method ofencoding multi-view video, and a method of decoding multi-view video asdescribed above may be assigned and stored.

A computer system embodied using a storage medium that stores a programfor executing a video encoding method and a video decoding method asdescribed above will now be described with reference to FIG. 22.

FIG. 22 illustrates a disc drive 26800 that records and reads a programby using a disc 26000. A computer system 26700 may store a program thatexecutes at least one of a video encoding method and a video decodingmethod according to an exemplary embodiment, in the disc 26000 via thedisc drive 26800. To run the program stored in the disc 26000 by thecomputer system 26700, the program may be read from the disc 26000 andbe transmitted to the computer system 26700 by using the disc drive26800.

The program that executes at least one of a video encoding method and avideo decoding method according to an exemplary embodiment may be storednot only in the disc 260 illustrated in FIG. 21 or 22 but also in amemory card, a ROM cassette, or a solid state drive (SSD).

A system to which the video encoding method and a video decoding methoddescribed above are applied will be described below.

FIG. 23 illustrates an entire structure of a content supply system 11000that provides a content distribution service. A service area of acommunication system is divided into predetermined-sized cells, andwireless default stations 11700, 11800, 11900, and 12000 are installedin these cells, respectively.

The content supply system 11000 includes a plurality of independentdevices. For example, the plurality of independent devices, such as acomputer 12100, a personal digital assistant (PDA) 12200, a video camera12300, and a mobile phone 12500, are connected to the Internet 11100 viaan Internet service provider 11200, a communication network 11400, andthe wireless default stations 11700, 11800, 11900, and 12000.

However, the content supply system 11000 is not limited to illustrationin FIG. 24, and devices may be selectively connected thereto. Theplurality of independent devices may be directly connected to thecommunication network 11400, not via the wireless default stations11700, 11800, 11900, and 12000.

The video camera 12300 is an imaging device, e.g., a digital videocamera, which is capable of capturing video images. The mobile phone12500 may employ at least one communication method from among variousprotocols, e.g., Personal Digital Communications (PDC), Code DivisionMultiple Access (CDMA), Wideband-Code Division Multiple Access (W-CDMA),Global System for Mobile Communications (GSM), and Personal HandyphoneSystem (PHS).

The video camera 12300 may be connected to a streaming server 11300 viathe wireless default station 11900 and the communication network 11400.The streaming server 11300 allows content received from a user via thevideo camera 12300 to be streamed via a real-time broadcast. The contentreceived from the video camera 12300 may be encoded using the videocamera 12300 or the streaming server 11300. Video data captured by thevideo camera 12300 may be transmitted to the streaming server 11300 viathe computer 12100.

Video data captured by the video camera 12300 may also be transmitted tothe streaming server 11300 via the computer 12100. The camera 12600 isan imaging device capable of capturing both still images and videoimages, similar to a digital camera. The video data captured by thecamera 12600 may be encoded using the camera 12600 or the computer12100. Software that performs encoding and decoding on video may bestored in a computer-readable recording medium, e.g., a CD-ROM disc, afloppy disc, a hard drive disc, an SSD, or a memory card, which may beaccessible by the computer 12100.

If video data is captured by a camera built in the mobile phone 12500,the video data may be received from the mobile phone 12500.

The video data may also be encoded by a large scale integrated circuit(LSI) system installed in the video camera 12300, the mobile phone12500, or the camera 12600.

According to an exemplary embodiment, the content supply system 11000may encode content data recorded by a user using the video camera 12300,the camera 12600, the mobile phone 12500, or another imaging device,e.g., content recorded during a concert, and transmit the encodedcontent data to the streaming server 11300. The streaming server 11300may transmit the encoded content data in a type of a streaming contentto other clients that request the content data.

The clients are devices capable of decoding the encoded content data,e.g., the computer 12100, the PDA 12200, the video camera 12300, or themobile phone 12500. Thus, the content supply system 11000 allows theclients to receive and reproduce the encoded content data. Also, thecontent supply system 11000 allows the clients to receive the encodedcontent data and decode and reproduce the encoded content data in realtime, thereby enabling personal broadcasting.

Encoding and decoding operations of the plurality of independent devicesincluded in the content supply system 11000 may be similar to those of avideo encoding apparatus and a video decoding apparatus according to anexemplary embodiment.

The mobile phone 12500 included in the content supply system 11000according to an exemplary embodiment will now be described in greaterdetail with referring to FIGS. 24 and 25.

FIG. 24 illustrates an external structure of the mobile phone 12500 towhich a video encoding method and a video decoding method are applied,according to an exemplary embodiment. The mobile phone 12500 may be asmartphone, the functions of which are not limited and a large part ofthe functions of which may be changed or expanded.

The mobile phone 12500 includes an internal antenna 12510 via which aradio-frequency (RF) signal may be exchanged with the wireless defaultstation 12000 of FIG. 24, and includes a display screen 12520 fordisplaying images captured by a camera 12530 or images that are receivedvia the antenna 12510 and decoded, e.g., a liquid crystal display (LCD)or an organic light-emitting diode (OLED) screen. The smartphone 12500includes an operation panel 12540 including a control button and a touchpanel. If the display screen 12520 is a touch screen, the operationpanel 12540 further includes a touch sensing panel of the display screen12520. The smartphone 12510 includes a speaker 12580 for outputtingvoice and sound or another type sound output unit, and a microphone12550 for inputting voice and sound or another type sound input unit.The smartphone 12510 further includes the camera 12530, such as acharge-coupled device (CCD) camera, to capture video and still images.The smartphone 12500 may further include a storage medium 12570 forstoring encoded/decoded data, e.g., video or still images captured bythe camera 12530, received via email, or obtained according to variousways; and a slot 12560 via which the storage medium 12570 is loaded intothe mobile phone 12500. The storage medium 12570 may be a flash memory,e.g., a secure digital (SD) card or an electrically erasable andprogrammable read only memory (EEPROM) included in a plastic case.

FIG. 25 illustrates an internal structure of the mobile phone 12500,according to an exemplary embodiment. To systemically control parts ofthe mobile phone 12500, including the display screen 12520 and theoperation panel 12540, a power supply circuit 12700, an operation inputcontroller 12640, an image encoding unit 12720, a camera interface12630, an LCD controller 12620, an image decoding unit 12690, amultiplexer/demultiplexer 12680, a recording/reading unit 12670, amodulation/demodulation unit 12660, and a sound processor 12650 areconnected to a central controller 12710 via a synchronization bus 12730.

If a user operates a power button and sets from a ‘power off’ state to a‘power on’ state, the power supply circuit 12700 supplies power to allthe parts of the mobile phone 12500 from a battery pack, thereby settingthe mobile phone 12500 in an operation mode.

The central controller 12710 includes a central processing unit (CPU),ROM, and random access memory (RAM).

While the mobile phone 12500 transmits communication data to theoutside, a digital signal is generated in the mobile phone 12500 undercontrol of the central controller 12710. For example, the soundprocessor 12650 may generate a digital sound signal, the image encodingunit 12720 may generate a digital image signal, and text data of amessage may be generated via the operation panel 12540 and the operationinput controller 12640. When a digital signal is transmitted to themodulation/demodulation unit 12660 under control of the centralcontroller 12710, the modulation/demodulation unit 12660 modulates afrequency band of the digital signal, and a communication circuit 12610performs digital-to-analog conversion (DAC) and frequency conversion onthe frequency band-modulated digital sound signal. A transmission signaloutput from the communication circuit 12610 may be transmitted to avoice communication default station or the wireless default station12000 via the antenna 12510.

For example, when the mobile phone 12500 is in a conversation mode, asound signal obtained via the microphone 12550 is transformed into adigital sound signal by the sound processor 12650, under control of thecentral controller 12710. The digital sound signal may be transformedinto a transformation signal via the modulation/demodulation unit 12660and the communication circuit 12610, and may be transmitted via theantenna 12510.

When a text message, e.g., email, is transmitted in a data communicationmode, text data of the text message is input via the operation panel12540 and is transmitted to the central controller 12610 via theoperation input controller 12640. Under control of the centralcontroller 12610, the text data is transformed into a transmissionsignal via the modulation/demodulation unit 12660 and the communicationcircuit 12610 and is transmitted to the wireless default station 12000via the antenna 12510.

To transmit image data in the data communication mode, image datacaptured by the camera 12530 is provided to the image encoding unit12720 via the camera interface 12630. The captured image data may bedirectly displayed on the display screen 12520 via the camera interface12630 and the LCD controller 12620.

A structure of the image encoding unit 12720 may correspond to that ofthe video encoding apparatus 100 described above. The image encodingunit 12720 may transform the image data received from the camera 12530into compressed and encoded image data according to a video encodingmethod employed by the video encoding apparatus 100 or the image encoder400 described above, and then output the encoded image data to themultiplexer/demultiplexer 12680. During a recording operation of thecamera 12530, a sound signal obtained by the microphone 12550 of themobile phone 12500 may be transformed into digital sound data via thesound processor 12650, and the digital sound data may be delivered tothe multiplexer/demultiplexer 12680.

The multiplexer/demultiplexer 12680 multiplexes the encoded image datareceived from the image encoding unit 12720, together with the sounddata received from the sound processor 12650. A result of multiplexingthe data may be transformed into a transmission signal via themodulation/demodulation unit 12660 and the communication circuit 12610,and may then be transmitted via the antenna 12510.

While the mobile phone 12500 receives communication data from theoutside, frequency transformation and ADC are performed on a signalreceived via the antenna 12510 to transform the signal into a digitalsignal. The modulation/demodulation unit 12660 modulates a frequencyband of the digital signal. The frequency-band modulated digital signalis transmitted to the video decoding unit 12690, the sound processor12650, or the LCD controller 12620, according to the type of the digitalsignal.

In the conversation mode, the mobile phone 12500 amplifies a signalreceived via the antenna 12510, and obtains a digital sound signal byperforming frequency conversion and ADC on the amplified signal. Areceived digital sound signal is transformed into an analog sound signalvia the modulation/demodulation unit 12660 and the sound processor12650, and the analog sound signal is output via the speaker 12580,under control of the central controller 12710.

When in the data communication mode, data of a video file accessed at anInternet website is received, a signal received from the wirelessdefault station 12000 via the antenna 12510 is output as multiplexeddata via the modulation/demodulation unit 12660, and the multiplexeddata is transmitted to the multiplexer/demultiplexer 12680.

To decode the multiplexed data received via the antenna 12510, themultiplexer/demultiplexer 12680 demultiplexes the multiplexed data intoan encoded video data stream and an encoded audio data stream. Via thesynchronization bus 12730, the encoded video data stream and the encodedaudio data stream are provided to the video decoding unit 12690 and thesound processor 12650, respectively.

A structure of the image decoding unit 12690 may correspond to that ofthe video decoding apparatus 200 described above. The image decodingunit 12690 may decode the encoded video data to obtain restored videodata and provide the restored video data to the display screen 12520 viathe LCD controller 12620, according to a video decoding method employedby the video decoding apparatus 200 or the image decoder 500 describedabove.

Thus, the data of the video file accessed at the Internet website may bedisplayed on the display screen 12520. At the same time, the soundprocessor 12650 may transform audio data into an analog sound signal,and provide the analog sound signal to the speaker 12580. Thus, audiodata contained in the video file accessed at the Internet website mayalso be reproduced via the speaker 12580.

The mobile phone 12500 or another type of communication terminal may bea transceiving terminal including both a video encoding apparatus and avideo decoding apparatus according to an exemplary embodiment, may be atransceiving terminal including only the video encoding apparatus, ormay be a transceiving terminal including only the video decodingapparatus.

A communication system according to the exemplary embodiment is notlimited to the communication system described above with reference toFIG. 24. For example, FIG. 26 illustrates a digital broadcasting systememploying a communication system, according to an exemplary embodiment.The digital broadcasting system of FIG. 26 may receive a digitalbroadcast transmitted via a satellite or a terrestrial network by usinga video encoding apparatus and a video decoding apparatus according toan exemplary embodiment.

Specifically, a broadcasting station 12890 transmits a video data streamto a communication satellite or a broadcasting satellite 12900 by usingradio waves. The broadcasting satellite 12900 transmits a broadcastsignal, and the broadcast signal is transmitted to a satellite broadcastreceiver via a household antenna 12860. In every house, an encoded videostream may be decoded and reproduced by a TV receiver 12810, a set-topbox 12870, or another device.

When a video decoding apparatus according to an exemplary embodiment isimplemented in a reproducing apparatus 12830, the reproducing apparatus12830 may parse and decode an encoded video stream recorded on a storagemedium 12820, such as a disc or a memory card to restore digitalsignals. Thus, the restored video signal may be reproduced, for example,on a monitor 12840.

In the set-top box 12870 connected to the antenna 12860 for asatellite/terrestrial broadcast or a cable antenna 12850 for receiving acable TV broadcast, a video decoding apparatus according to an exemplaryembodiment may be installed. Data output from the set-top box 12870 mayalso be reproduced on a TV monitor 12880.

As another example, a video decoding apparatus according to an exemplaryembodiment may be installed in the TV receiver 12810 instead of theset-top box 12870.

An automobile 12920, including an appropriate antenna 12910, may receivea signal transmitted from the satellite 12900 or the wireless defaultstation 11700. A decoded video may be reproduced on a display screen ofan automobile navigation system 12930 built in the automobile 12920.

A video signal may be encoded by a video encoding apparatus according toan exemplary embodiment and may then be stored in a storage medium.Specifically, an image signal may be stored in a DVD disc 12960 by a DVDrecorder or may be stored in a hard disc by a hard disc recorder 12950.As another example, the video signal may be stored in an SD card 12970.If the hard disc recorder 12950 includes a video decoding apparatusaccording to an exemplary embodiment, a video signal recorded on the DVDdisc 12960, the SD card 12970, or another storage medium may bereproduced on the TV monitor 12880.

The automobile navigation system 12930 may not include the camera 12530,the camera interface 12630, and the image encoding unit 12720 of FIG.26. For example, the computer 12100 and the TV receiver 12810 may not beincluded in the camera 12530, the camera interface 12630, and the imageencoding unit 12720 of FIG. 26.

FIG. 27 illustrates a network structure of a cloud computing systemusing a video encoding apparatus and a video decoding apparatus,according to an exemplary embodiment.

The cloud computing system may include a cloud computing server 14000, auser database (DB) 14100, a plurality of computing resources 14200, anda user terminal.

The cloud computing system provides an on-demand outsourcing service ofthe plurality of computing resources 14200 via a data communicationnetwork, e.g., the Internet, in response to a request from the userterminal. Under a cloud computing environment, a service providerprovides users with desired services by combining computing resources atdata centers located at physically different locations by usingvirtualization technology. A service user does not have to installcomputing resources, e.g., an application, a storage, an operatingsystem (OS), and security, into his/her own terminal in order to usethem, but may select and use desired services from among services in avirtual space generated through the virtualization technology, at adesired point of time.

A user terminal of a specified service user is connected to the cloudcomputing server 14000 via a data communication network including theInternet and a mobile telecommunication network. User terminals may beprovided cloud computing services, and particularly video reproductionservices, from the cloud computing server 14000. The user terminals maybe various types of electronic devices capable of being connected to theInternet, e.g., a desktop PC 14300, a smart TV 14400, a smartphone14500, a notebook computer 14600, a portable multimedia player (PMP)14700, a tablet PC 14800, and the like.

The cloud computing server 14000 may combine the plurality of computingresources 14200 distributed in a cloud network and provide userterminals with a result of the combining. The plurality of computingresources 14200 may include various data services, and may include datauploaded from user terminals. As described above, the cloud computingserver 14000 may provide user terminals with desired services bycombining video data default distributed in different regions accordingto the virtualization technology.

User information about users who have subscribed to a cloud computingservice is stored in the user DB 14100. The user information may includelogging information, addresses, names, and personal credit informationof the users. The user information may further include indexes ofvideos. Here, the indexes may include a list of videos that have alreadybeen reproduced, a list of videos that are being reproduced, a pausingpoint of a video that was being reproduced, and the like.

Information about a video stored in the user DB 14100 may be sharedbetween user devices. For example, when a video service is provided tothe notebook computer 14600 in response to a request from the notebookcomputer 14600, a reproduction history of the video service is stored inthe user DB 14100. When a request to reproduce this video service isreceived from the smartphone 14500, the cloud computing server 14000searches for and reproduces this video service, based on the user DB14100. When the smartphone 14500 receives a video data stream from thecloud computing server 14000, a process of reproducing video by decodingthe video data stream is similar to an operation of the mobile phone12500 described above with reference to FIG. 24.

The cloud computing server 14100 may refer to a reproduction history ofa desired video service, stored in the user DB 14100. For example, thecloud computing server 14100 receives a request to reproduce a videostored in the user DB 14100, from a user terminal. If this video wasbeing reproduced, then a method of streaming this video, performed bythe cloud computing server 14000 may vary according to the request fromthe user terminal, i.e., according to whether the video will bereproduced, starting from a start thereof or a pausing point thereof.For example, if the user terminal requests to reproduce the video,starting from the start thereof, the cloud computing server 14000transmits streaming data of the video starting from a first framethereof to the user terminal. If the user terminal requests to reproducethe video, starting from the pausing point thereof, the cloud computingserver 14000 transmits streaming data of the video starting from a framecorresponding to the pausing point, to the user terminal.

In this case, the user terminal may include a video decoding apparatusas described above with reference to FIGS. 1A to 20. As another example,the user terminal may include a video encoding apparatus as describedabove with reference to FIGS. 1A to 20. Alternatively, the user terminalmay include both the video decoding apparatus and the video encodingapparatus as described above with reference to FIGS. 1A to 20.

Various applications of a video encoding method, a video decodingmethod, a video encoding apparatus, and a video decoding apparatusaccording to exemplary embodiments described above with reference toFIGS. 1A to 20 have been described above with reference to FIGS. 21 to27. However, methods of storing the video encoding method and the videodecoding method in a storage medium or methods of implementing the videoencoding apparatus and the video decoding apparatus in a deviceaccording to various exemplary embodiments are not limited to theexemplary embodiments described above with reference to FIGS. 21 to 27.

While the exemplary embodiments have been particularly shown anddescribed, it will be understood by those of ordinary skill in the artthat various changes in form and details may be made therein withoutdeparting from the spirit and scope the present application as definedby the following claims.

What is claimed is:
 1. A method for decoding a video, the methodcomprising: obtaining, from a bitstream, information about a location ofa column boundary of the tile; determining tiles, including a currenttile, based on the information about the location of the column boundaryof the tile; and obtaining, based on split information of a coding unit,at least one coding unit from a maximum coding unit included in both thecurrent tile and a current slice segment, wherein all maximum codingunits in the current slice segment are included in the current tile, andwhen the split information indicate a split at a current depth, a codingunit of the current depth is split into coding units of a lower depth.2. The method of claim 1, wherein the maximum coding unit ishierarchically split into at least one coding unit of a depth accordingto the split information, the depth including at least one of a currentdepth and a lower depth, when the split information indicates a split atthe current depth, the coding unit of the current depth is split intofour square coding units of a lower depth independently from neighboringcoding units, and when the split information indicates a non-split atthe current depth, at least one prediction unit is obtained from thecoding unit of the current depth and at least one transformation unit isobtained from the coding unit of the current depth.
 3. The method ofclaim 1, wherein the slice segment includes at least one maximum codingunit according to a raster scan order, the slice segment is contained ina single NAL (Network Adaptation Layer) unit, and the slice segment doesnot exceed a boundary of the current tile.
 4. The method of claim 1,wherein first information is obtained from the bitstream, the firstinformation indicating whether the current slice segment is an initialslice segment in a picture; when the first information indicates thatthe current slice segment is not the initial slice segment, secondinformation is obtained from the bitstream, the second informationindicating whether the current slice segment is a dependent slicesegment; when the second information indicates that the current slicesegment is the dependent slice segment, header information of thecurrent slice segment is obtained based on header information of otherslice segment. when the second information indicates that the currentslice segment is an independent slice segment, the header information ofthe current slice segment is obtained from the bitstream.